An incremental solution for developing knowledge-based software: its application to an expert system for isokinetics interpretation

An incremental solution for developing knowledge-based software: its application to an expert system for isokinetics interpretation

Expert Systems with Applications PERGAMON Expert Systems with Applications 18 (2000) 165–184 An incremental solution fo...

4MB Sizes 0 Downloads 97 Views

Expert Systems with Applications PERGAMON

Expert Systems with Applications 18 (2000) 165–184

An incremental solution for developing knowledge-based software: its application to an expert system for isokinetics interpretation F. Alonso*, J.L. Fuertes, L. Martı´nez, C. Montes Facultad de Informa´tica, UPMC campus de Montegancedo, 28660 Boadilla del Monte, Madrid, Spain

Abstract Many researchers working in the field of knowledge engineering (KE) are now concerned with identifying a model suitable for developing knowledge-based software and, especially, expert systems (ES). It is important to find a standard model that meets current needs and incorporates techniques successfully implemented in SE (object- or event-orientation, etc.), which are also of keen interest in KE. In this paper, we present an iterative and incremental solution for developing ES, according to which the system domain is derived naturally from the problem domain, thus surmounting the problems now involved in the transition from the conceptual model of the problem to the formal model of the system. As compared with conventional development models, this solution encompasses five main tools, which are: X Use cases with their respective actor interaction diagrams and activity flow diagrams in order to specify the expert system. X The concept dictionary, which allows knowledge engineers to define, bound and select the meaning of each concept used by experts. X The static conceptual model, which provides an overview (concepts and their relations) of the expert system (ES) modelled. X The control and process model, which models the knowledge and metaknowledge used by the expert to attain a goal. X An object-oriented metamodel, which outputs the formal knowledge model, providing an efficient, reusable, extendible and easy-toimplement ES architecture. To demonstrate the robustness of this solution, we describe how it was applied to an ES that interprets the graphs output by an isokinetics machine for a blind person. An isokinetics machine assesses the strength of the muscles of the leg, arm, etc. 䉷 2000 Elsevier Science Ltd. All rights reserved. Keywords: Knowledge engineering; Expert system; Isokinetics; ES development methodologies; Knowledge-based systems

1. Introduction Software process development is represented in traditional knowledge engineering (KE) methodologies by a life-cycle model that specifies the order in which the different development phases should be executed. The important thing for the early methodologies was to implement the acquired knowledge. This knowledge was hardly structured at all, and an error in the requirements meant that the entire process had to be repeated from the beginning. The secondgeneration methodologies, including KADS, had a waterfall life cycle, whose primary feature was the feedback between the individual stages. This meant that each time a phase (defined in terms of the activities to be performed and the products to be output) ended, it was possible to check that it was error free and met the original requirements. Otherwise, developers went back to phase in which the error had arisen. * Corresponding author. Tel.: ⫹34-91-352-25-46; fax: ⫹34-91-352-63-88. E-mail address: [email protected] (F. Alonso).

The waterfall model obliged customers to provide all the requirements at the start of the projects and ended with an implementation phase that ought to meet user needs. This traditional life cycle model aimed at well-defined problems is not applicable to the problems that we deal with in KE, which can be characterised as poorly defined and have all or some of the following peculiarities: • Poorly structured—either they cannot be described in terms of well-defined functions or there is no computational algorithm capable of solving them. • Subjective requirements—requirements that cannot be extracted from public knowledge. • Context dependent—for example, natural language. • Conditions of uncertainty—as a result of data, information or knowledge incompleteness, inaccuracy or inconsistency, which prevents direct development of a software system. This forced researchers to look in other directions for lifecycle models, and Boehm came up with the spiral life cycle

0957-4174/00/$ - see front matter 䉷 2000 Elsevier Science Ltd. All rights reserved. PII: S0957-417 4(99)00060-3


F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184

Fig. 1. Spiral/conical model of a KE system life cycle (top view).

model (Boehm, 1988), which is composed of several cycles. Each cycle starts by identifying the objectives of the intermediate product developed within the cycle in question, alternatives for implementing the product (reuse of A, design of B, purchase of C, etc.) and the factors (interfaces, costs, etc.). Then this is all evaluated, and the possible risks and alternatives are singled out. Finally, the product is developed. Being based on a prototyping system that

includes risk analysis during each cycle, this method is better than its predecessors. Nevertheless, all the system requirements have to have been properly addressed by the end of the spiral traced by the life cycle, as it does not provide for the incorporation of new requirements once the system is operational. This, however, is very often just what is needed by a knowledge-based system (KBS), as domain knowledge increases with the passage of time.

F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184


Phase III—Commercial system. Phase IV—Technology transfer and system maintenance. In this paper, we will primarily focus on the first two phases (I and II), which show the peculiar features of the solution. 2.1. Application identification (Phase I) The first point to be considered before starting to develop an ES (for example, an automated system for disease diagnosis) is to define application goals and then determine whether the task can be addressed taking a KE approach. If so, the characteristics of the problem are defined and the requirements that will mark out the problem-solving process are specified. This phase is divided into three stages: requirements plan, application evaluation and selection and application definition.

Fig. 2. Spiral/conical model of a KE system life cycle (side view).

When a KBS and particularly an ES is being developed, such as a medical ES for automated disease diagnosis, instantaneous system development may largely fit in with Boehm’s spiral model. However, the expert’s knowledge of the disease increases with time, and this new knowledge must be input into the ES at some point, giving rise to a new development cycle with similar characteristics to the preceding phase. We and other colleagues proposed a 3D, conical/spiral-type life cycle (Alonso & Pazos, 1990; Alonso et al., 1992) to deal with this problem. This 3D spiral model is founded on Spengler’s idea (Spengler, 1918), taken up again by Bartholomew (1988), to show the historical cycles of terrestrial geology, and Boehm’s spiral model. Each spiral on the plane defines the development of the ES from the knowledge of the expert at a given time. The third dimension would be perfective development of the ES at a later stage using the new knowledge acquired by the expert. This would lead to another spiral life cycle with shorter stages and smaller prototypes located at another level, i.e., a spiral cone would be obtained as shown in Figs. 1 and 2. 2. Incremental solution to ES development applying the 3D spiral model For developing an ES based on this 3D-spiral life cycle model, we propose the following incremental solution, composed of four phases, which define the milestones in ES development: Phase I—Application identification. Phase II—Operational prototype development.

2.1.1. Requirements plan The first step to be taken by knowledge engineers is to analyse the system as in software engineering, i.e., to identify customer needs and describe system goals: information to be input, information to be output, functionality requirements, etc., and the resources needed for system development. 2.1.2. Application evaluation and selection This methodology provides two actions for this stage, which, in KE, can be defined as the feasibility study: (a) evaluate the task from a KE viewpoint and (b) quantify this assessment to find out whether the task surpasses a threshold that recommends development or to select the best task, where there is more than one candidate. It is by no means easy to give a general description of the characteristics that define whether or not an ES can perform a task. The solution applies a test that measures four main aspects or dimensions (Go´mez, Juristo & Pazos, 1997): • Plausibility—an ES can be developed and built if there are real experts. • Justification—ES construction is justified if there is a shortage of human experts. • Fitness—it is right to build an ES if the problem basically involves symbolic manipulation, is solved by heuristics and the knowledge required is not elementary. • Success—the ES will be successful if managers are convinced of the importance and efficiency of this technology. A series of characteristics that it would be essential (vital for task performance) or desirable (positive for task performance) for managers and/or users, experts and task to have are defined for each dimension. Each characteristic is evaluated using Boolean (yes, no), numerical (on a scale of 0–10) or fuzzy values (absolutely, very, fairly, not very, not at all) to rate whether or not the characteristic is present. These characteristic values are represented in fuzzy intervals, entering the four interval points to represent each value


F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184

Fig. 3. Use case-based model of a job adaptation system for the disabled.

so that they can then be used as if they were numerical values. The Boolean values will be represented in fuzzy intervals as follows: the value no as (0,0,0,0) and the value yes as (10,10,10,10). Each dimension is evaluated by calculating the arithmetic mean and harmonic mean of the set of fuzzy intervals of its characteristics separately, and then taking the arithmetic mean of the two intervals obtained, i.e. ri X

Vci ˆ

Wik 1 kˆ1 1 ⫹ ri 2 X 2 Wik =Vik kˆ1

ri X

Wik ⴱ Vik

kˆ1 ri X



where Vci is the total value of the application in a given dimension, Vik the value of characteristic k in dimension i, Wik the weight of characteristic k in dimension i, and ri is the number of characteristics in dimension i. The above formula produces four fuzzy intervals (one for each dimension), whose arithmetic mean outputs a fuzzy value, which will be the final result of the evaluation. The task is selected if it scores higher than the threshold value, and if there are several candidates, the highest scoring task will be selected.

2.1.3. Application definition This stage actually puts together system specifications (Alonso, Barreiro, Fuertes, Martı´nez & Montes, 1999a). While the requirements plan describes some mini specifications of the application, which serve as a basis for task evaluation and selection, it is at this stage that the specifications are completed with all the available knowledge about the system. KE has tended to overlook this development phase, using the excuse that preliminary requirements are vague, because of the intrinsic characteristics of the problem types addressed by this branch of engineering. In SE, however, this phase has come to be a decisive milestone in the development of any project to the point that full specifications are often produced of all the requirements of the target system. It is obviously impossible for KE to be as accurate as SE, precisely because of the problem type addressed. Nevertheless, the system requirements should be modelled so as to define the functions that it is to provide unambiguously, even if it is not possible to describe how they are to be performed. In this stage, then, developers output an overall functional description, a list of success criteria and a rough estimate of resources. In other words, the application is defined from the viewpoint of the system on the basis of: • Functional requirements (data types to be processed, operations, desired outputs). • Interface requirements (user and expert interfaces, hardware interfaces, interfaces with other products or systems). • Success criteria (test cases and application validation, etc.). • Material and human resources (milestones and schedule, cost/benefit analysis, etc.). We recommend that use cases, which describe the services required by the system, be employed as a means of modelling system requirements and outputting appropriate specifications. A use case defines the entire sequence of interactions between the user and system during a system activity that provides a particular service for the user (Yourdon et al., 1995).

Fig. 4. Adaptation for the blind use case actors interaction diagram.

F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184


Fig. 6. The essential software process.

Fig. 5. Adaptation for the blind use case activities flow diagram.

Fig. 3 describes all the use cases of a job adaptation ES for the disabled. Two actors participate in the system: the disabled person and the instructor who will use the system to adapt the job. Each use case is represented by an ellipse, each actor by a rectangle and the arrows indicate the flow of communication between the actor and the use case. Each use case is modelled by means of an actors interaction diagram (UCAID) and an activity flow diagram (AFD). The UCAID provides a declarative description of the use case, whereas the AFD supplies a procedural description. Fig. 4 illustrates the use case “Job Adaptation for the Blind”, using an actors interaction diagram, and Fig. 5 shows this same adaptation, using an activities flow diagram. The UCAID shown in Fig. 4 involves two actors (the blind person and the instructor), plus the expert system that acts at this level as an actor. Four activities (provide data on blindness, enter data on blindness and job, define suitable aids and appliances for the job and degree of blindness in question, provide aids and appliances for the job in question) describe the use case. Some activities will be performed manually (i.e., any carried out by the blind person and the instructor) and others will be computerised (i.e., any in which the expert system is involved). It is the computerised activities that will be used to specify the software system. The AFD shown in Fig. 5 is a procedural description of the same use case. Here, the activities are depicted by hexagons, the alternative decisions by triangles, the root of the use case by an ellipse and the end of the use case by a dot. The information output by each activity is specified at the side of the lines that connect the activities. UCAIDs do not describe procedural structures of the use case, for example,

specification of repetitive activities or the possibility of part of the process being performed at random or simultaneously. This issue is addressed using AFDs that describe this type of decisions. UCAIDs are an appropriate technique for linking requirements and system operations. When the system has been developed, UCAIDs can be used to check that the actors and operations designed provide the specified system functionality. The AFD surpasses the UCAID in that it describes the sequences of activities addressed; however, it does not describe the interaction between the actor agents, which is what is used to model the system. After application definition, the knowledge engineers, experts, customers and users consider that the project scope has been satisfactorily profiled, functionality, performance and interfaces have been coherently defined, the task environment and development risk analyses justify the system and that they all have the same perception of system goals. In any case, it is important to always bear in mind that the preliminary specifications of an ES are usually incomplete, imprecise, inconsistent and sometimes even contradictory, which means that several prototyping phases will be required before a real and full definition is attained. 2.2. Operational prototype development (Phase II) In conventional systems analysis, the system is first viewed from the user perspective and then the problem is looked at from the computer viewpoint. More formally, as Blum (1992) pointed out as regards software development (Fig. 6), the development of an ES can be described as a set of transformations …T1 ; T2 ; T3 † from task identification in the problem domain to a software product that operates in the system domain. Fig. 6 identifies two modelling activities: conceptual modelling, in the problem domain, which models how the software is to respond to system specifications, and formal modelling, in the system domain, which starts once a conceptual model exists and models how the computer must respond to those specifications. The conceptual model determines validity and the formal model, correctness. It is essential to establish that a conceptual model is correct, as formal models may uncover errors that invalidate the model. The two models, which enable production of a prototype software system that seeks to model the real world, are developed in phase II. Firstly, the conceptual model of the ES is defined. The conceptual model addresses the manner of inferring and controlling knowledge. Later, this


F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184

Fig. 7. Concept dictionary.

knowledge is formalised using representation and inference techniques to output the formal model of this knowledge from the viewpoint of the computer. This methodology recommends an iterative and incremental development process based on prototyping that allows gradual system development, because, as pointed out earlier, the exact specifications of what can be done and how it can be done are not known at the start of ES construction, and many problems first come to light during prototype development. Most ESs start out life as small programs designed to demonstrate ES feasibility (demonstration prototypes). These prototypes address part of the problem that the ES is to solve. Apart from demonstrating to the customer that the technology implemented is sound, this prototype is used to verify the problem definition, scope and domain representation. A demonstration prototype may contain some 100 rules and operate on several test cases (three or four), and will only take a few months to develop (Alonso et al., 1999a). Prototyping continues to add new knowledge, extending and refining the ES, up until an operational prototype is built that includes all the existing knowledge required for full system operation (see Fig. 1). Prototype development is divided into four stages. 2.2.1. Solution conception During the application definition stage of phase I, the problem centred on obtaining specifications of the entire system, both with regard to the manual tasks (e.g., defining a form to be filled in by the blind person) and software system activities (e.g., outputting aids and appliances for a job). Phase II centres specifically on ES software development, which means that this stage seeks to output the specifications of the software system to be developed. For this purpose, the UCAID and AFD activities that interact with the software system actor are analysed, as these are the activities that will allow ES development from the software viewpoint. A list will include a functional and operational classifi-

cation of the activities of each use case that is to be included in the ES. Also, the interfaces, success criteria and resources required for ES development will be specified. 2.2.2. Knowledge acquisition and conceptualisation This stage produces the conceptual model of the knowledge before formalisation and is divided into two basic activities that are developed jointly in a cyclical manner. First, public and private knowledge must be acquired and, second, this knowledge must be structured for later use, i.e., it must be conceptualised in order to define the properties of the concepts and the default values of their attributes. This is a cyclical acquisition-conceptualisation process, which continues until the conceptual knowledge model is obtained, after which the knowledge can be formalised. Knowledge acquisition. The knowledge contained in an ES has several sources; however, an expert’s knowledge about the real system domain is dominant. It is precisely the acquisition of this knowledge that constitutes a knowledge engineer’s main problem. While there are some machine learning tools around that can be used as aids for extracting knowledge from examples (Montes, 1994), there is as yet no efficient automated method for acquiring expert knowledge, which means that a knowledge engineer’s work is a craft of vital importance. A knowledge engineer should acquire four types of knowledge (Alonso, Juristo, Mate´ & Pazos, 1996): structural (identifying domain objects, basic relations, possible solutions, etc.), heuristic (expertise enabling the expert to bound the search space), epistemological (applied by means of common sense, not on the basis of expertise, to solve the problem) and conceptual (making it possible to model objects, relations, facts and other domain elements, including strategy-, causal- and formal-based reasoning diagrams). The recommended techniques for acquiring this knowledge, especially from experts, are (Go´mez et al., 1997): • Interviews with experts (structured and non-structured).

F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184


Fig. 8. Static conceptual model for adapting a job for a disabled person.

• Task and protocol analysis on the basis of expert problem-solving behaviour. • Repertory grid and multidimensional scaling, showing how a particular concept set is structured. • Delphi method. Although it is at this point that most of the ES knowledge is acquired, it is important to note that the knowledge acquisition process continues, decreasingly, right up to the system formalisation phase. Conceptualisation. The activity of knowledge acquisition outputs unstructured knowledge that must be organised if it is to be used later to define concepts and their properties and to output the conceptual model. This knowledge structuring is what is known as conceptualisation or epistemological design and outputs an external representation of the knowledge (conceptual model) that is independent of its implementation (formal model). Indeed, conceptualisation, as a knowledge transformation process, restructures the knowledge contained in the acquisition output (normally, a working document containing words, numbers and diagrams, which set out the unstructured knowledge acquired) and shapes it so that it can be used to build the ES. The purpose of conceptualisation is really to get and model the expert’s strategic, tactical and factual knowledge about the system, considering that: • Strategic knowledge (what to do) specifies the sequence of steps to be taken by the ES to execute a task. • Tactical knowledge (how to infer) specifies how the ES can routinely use known facts and currently valid hypotheses about the case to infer new facts and hypotheses. • Factual knowledge (what is known) specifies what is, or

is thought to be, true about the world generally and about the particular case for which the task is being executed. Conceptualisation as proposed by this approach is composed of the following six stages (Alonso, Juristo & Pazos, 1995; Alonso, Fuertes, Martı´nez & Montes, 1997): • Identify the factual knowledge and metaknowledge by identifying concepts (abstractions, e.g., disability) or particular objects (e.g., a blind person), their structures of association, classification or generalisation, which define inheritance relations of the type “is_a”, and assembly or composition, which define membership relations such as “is_part_of”, and their associated values, and the attributes defining each concept, all of which are entered in the concept dictionary (Fig. 7). The concept dictionary is a powerful tool that enables knowledge engineers to understand, relate and bound the meaning of each concept used by the expert, i.e., structure expert knowledge. For example, taking the concept disabled_person, knowledge engineers will be able to consult the dictionary and find out that it is defined by the attributes disability_type, disability_degree, usual_aids/appliances, etc., is associated with aids/appliances and job_type and is classified as blind, deaf, tetraplegic, etc. (see Fig. 8). • Define the static conceptual model or concept map, specifying the connecting lines between related concepts, their multiplicity (e.g., 0..1, 1..1 or 0..N relations between concepts) and their role (is a connection compulsory or optional?), checking out special cases. The aim is to get a logical and static model of the ES based on the concepts identified and the dependencies between concepts. This will give knowledge engineers an overview of the ES


F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184

Fig. 9. Process and control model to attain a goal.

modelled and enable them to examine its structure, coherence and any conceptual knowledge that has been omitted. Fig. 8 shows an example of the static conceptual model for adapting a job for a disabled person. It indicates that a disabled person may be blind, deaf, etc., and use one or several aids and appliances to perform his or her job. One job type may be performed by more than one disabled person and require one or more aids and appliances, whereas one particular aid and appliance may or may not be used by a disabled person and used in one or more types of job. The conceptual model depicted in Fig. 8 shows three relationships of association: disabled person and aids and appliances (uses), disabled person and job type (performs), job type and aids and appliances (requires). It also includes one classificational relationship (inheritance) between the disabled person and blind, deaf, …, tetraplegic). • Model and structure system strategic and tactical knowledge to establish the process and control model. Knowledge engineers must extract and define the expert knowledge and metaknowledge from the static conceptual model that will enable them to model the process for goal decision making. The system usually used to identify this knowledge and metaknowledge is to get the expert to divide this goal decision into a series of subdecisions that must be identified by the knowledge engineers and confirmed by the experts as correct. The

output of this activity is specified in the process and control model. A functional decomposition diagram should be used to define expert strategic knowledge. Its root will be a decision and its leaves the consecutive subdecisions to be made. This means that the expert knowledge and metaknowledge needed to achieve a goal can be modelled in a structured and clearer manner. The process and control model represents the strategic knowledge according to what is known as a mosque organisation: non-leaf tasks represent control activities, while leaf tasks represent inference activities. Fig. 9 shows the process and control model for establishing whether it is feasible for the disabled person to perform a job type, which is divided into five tasks. These activities and their respective models constitute the analysis stage of conceptualisation, where all the information is broken down and classed according to its usefulness and type. This information must then be combined in an exercise of synthesis so that the correspondence between each factual, tactical or strategic item can be identified and the models built can be checked for validity. Synthesis is the nexus between the problem and the system domain. • Define the knowledge map, which describes the processes of inferring values from concept attributes, representing the links between the attributes and the inferred values and is an ideal synthesis tool for validating the process

Fig. 10. Knowledge map for performing a job type.

F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184


Table 1 Disabled, aid/appliance, job type model services Concept

Service type

Service name

Disabled person

Specify disability type Specify degree of disability Specify known aids and appliances … Specify characteristics of the aid/appliance Specify job type for which it is used Specify whether or not the aid/appliance is available … Job name Specify aids/appliances needed depending on disability Specify substitute aids/ appliances depending on disability …

disability_type ( ) disability_degree ( ) known_aids/appliances ( )


Job type

and control model. The knowledge map is like a mental map that represents the connections made by the brain when analysing facts about something. Naughton (1989) points out that knowledge maps: ◦ Are a powerful means of depicting how much and what type of expert knowledge has been gathered. ◦ Embed subjects that indicate the layers of knowledge gathered from the expert, which means that all the previous analytical steps can be traced back until it is possible to determine exactly whence the knowledge was obtained and in what circumstances. The knowledge map is built by:

aid/appliance_characteristics ( ) jobtype_aid/appliance ( ) available_aid/appliance ( )

job_type ( ) necessary_aids/ appliances_disability ( ) substitute_aids/ appliances_disability ( )

◦ Identifying the decisions made by the expert about the tasks. ◦ Designing the goal decision block, adding attributes around the goal decision and also incorporating the attributes used to infer or calculate the values of attributes to deduce the goal decision. The knowledge map is the main external representation of inference formalisms and visually transmits the values of the attributes and their relations in a clear and concise manner. Fig. 10 pictures a partial knowledge map for performing a job type (goal decision), which corresponds with the process of attaining the goal, shown in Fig. 9.

Fig. 11. Formal ES metamodel.


F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184

• Identify and define the services of each concept. A service is the potential activity offered by a concept when required (e.g., specify the aids and appliances familiar to the disabled person). The central idea behind identifying services is to be able to define the behaviour required of each concept. This step involves identifying what services should be provided by each concept in order to develop the process and control model output in the preceding stage. So, a service will define an expert subdecision (subtask) and will often involve inferring an attribute value. For example, Table 1 identifies some of the services of the concepts disabled person, aid and appliance and job type, which would make it possible to achieve the goal according to the process and control model shown in Fig. 9. When identifying concept services, it often happens that a concept appears to be isolated in the control and process model (i.e., it has no associated service) or a service appears not to apply to any concept. In such cases, it is important to analyse whether: ◦ The concept identification is incomplete or anything relevant has been omitted, or the association, classification and assembly structures are inappropriate, in which cases the static conceptual model should be revised. ◦ The decisions taken by the experts to reach the final goal have been incorrectly identified by the knowledge engineers, in which case the dynamic process and control model should be revised. Service definition involves making a declarative or procedural description of each service of each concept identified earlier. Obviously, the name of all the services identified and a brief description of their action will be entered into the concept dictionary. • Reuse the information and refine the system. At this point of the methodology, the following conceptualisation subproducts will have been output: ◦ concept dictionary, with attributes, services and dependencies between concepts; ◦ system static conceptual model; ◦ dynamic process and control model; and ◦ knowledge map. When the system is analysed globally using these items, some attributes and services are found to be common to several concepts and can be combined in a higher-level concept. This provides for information reuse and ES refinement. So, in the conceptual model shown in Fig. 8 (disabled person, aids/appliances, job_type), the concept disabled person becomes a non-instantiable concept that encompasses the attributes and services common to blind, deaf, tetraplegic, etc., which will be instantiable concrete concepts that interact in the system.

2.2.3. Knowledge formalisation This stage involves two basic activities. Firstly, the

formalisms that are to represent the knowledge must be selected (under this methodology, the knowledge is formalised in an object-oriented model, as the concepts are classes of objects) and, secondly, a formal model of the ES must be completed, applying a modular system structure that must match the Solution Conception. It is important to note that, under this methodology, the static conceptual model defines concepts, and the process and control model defines the services of these concepts and the inference engine control module, which means that the ES would be modelled as shown in the metamodel depicted in Fig. 11. The formal metamodel is composed of: concepts, concept base, control module, inference engine with its working memory, and man/machine interfaces. It specifies that: (a) the concept base is an aggregate of concepts, in which each concept can belong to several concept bases; (b) the control module can use various concept bases, different interfaces and is dependent on the inference engine; (c) the inference engine can work with various control modules, explore several concept bases and use its working memory to represent the current status of knowledge in the problem-solving process. This metamodel allows the instantiation of a specific formal model, suited to the characteristics of each particular problem, in which each concept is implemented as a class. The Concept Base is a container class of all the concepts instantiated during system execution. Control, whose job is to organise the domain knowledge of the application in order to assure satisfactory problem solving, is implemented as a class that instantiates the concept base, the inference engine and the interface. It includes the strategic expert knowledge for problem solving by means of methods, calling out to the inference engine as required to infer the values of the attributes of the instantiated concepts. The Inference Engine is responsible for reasoning and its goal is to interpret the knowledge contained in the concept base in response to queries from the control module. It has a domain-independent structure, i.e., it is independent of the knowledge contained in the concept base. In the model, it is implemented by means of a class, in which each inference strategy (pattern matching, forward reasoning, backward reasoning, etc.) is a specific class method. This makes the system very versatile as it can use the most appropriate strategy for solving each particular problem. The working memory is linked to the inference engine that creates it dynamically when it is instantiated. It contains the problem-specific data supplied by control and the partial results obtained during the inference process. The man/machine interface has two missions: (a) to transmit the information and knowledge supplied by the experts/ knowledge engineers to the system, and (b) to interact with the user both to ask for information and give explanations on the steps taken during the reasoning process, in both cases by means of control module queries. It is implemented in the model by means of a class with two functionalities depending on its mission ((a) or (b)). In case (a), the main method

F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184

of the class will be executed and will update the concept base and its concepts. In case (b), handle event will be executed and will query the control model for the information or explanation requested by the user.

2.2.4. Implementation The implementation phase is concerned with transforming the formalised knowledge obtained in the previous phase into source code. It encompasses a series of essential tasks, like selecting the tool or the programming language to be used and defining the user interface. The choice of programming tool or language will strongly condition the remainder of the development work, whereas the design of the user interface will have a major impact on user acceptance and system use. Generally, there are three different alternatives for implementation: 1. Use one of the many specialised tools on the market (e.g., Kappa, OPS5, ART-IM, Guru, etc.). The drawback of this solution is generally that the resulting product is not very efficient, not very flexible and not usually feasible when the ES is part of a bigger application, as the resulting code often cannot be compiled with the other application code. 2. Use one of the many general-purpose programming languages (e.g., C, C⫹⫹, Object Pascal, CLOS, Prolog, etc.). This solution has a series of important drawbacks, such as slower and more difficult development and higher costs (if a programming language is used, some of the built-in tool mechanisms have to be constructed by the programmer; inference engine development, for instance). However, it has some significant benefits, such as, the code output using a programming language is considerably more efficient than the code produced by a tool, and the resulting systems are more flexible than any produced using tools. 3. Use component libraries. This solution is usually more advantageous than the other two options, as it allows production of a system that is more flexible and efficient than one produced by a tool (Alonso, Fuertes, Martı´nez & Montes, 1999b), albeit less so than a programming language, while development time is comparable to when a tool is used. Therefore, the use of libraries, such as CLASER (a generic object-oriented library for ES construction, in which each ES element is a template class) (Alonso, Fuertes, Lo´pez & Montes, 1994; Garcı´a, 1993) or similar, which are reusable, extendible and independent of a complex, large and costly integrated tool, is a very good option for developing an ES. The selection of one option or another must be studied carefully by knowledge engineers in order to ensure that it is suited to the specific characteristics of the ES under development.


2.2.5. Validation and evaluation Validation involves assessing whether the features of the system comply with customer expectations. This means that the system has to be evaluated by the expert and the modifications identified by the expert have to be implemented by the knowledge engineer. In this stage, the expert provides information about correct solutions to the problem, which are compared with the system outputs. Test cases sometimes need to be divided into subclasses: (a) Test cases that could be completely solved by the expert. (b) Test cases that could be partially solved by the expert. (c) Test cases that could not be solved by the expert. Nevertheless, the test cases chosen should represent all the problem types that the ES should be capable of solving, thus affording a detailed results analysis. This means that the system can be evaluated against the original goals. Without the validation phase, it is impossible to properly judge the impact of the system. Validation should be performed from the start to the finish of the project, because it is important to detect software faults as early on as possible. For instance, an error in conceptualisation that is discovered in the implementation phase will entail a lot of changes, because the error has been propagated through three phases, and the conceptualisation, formalisation and implementation will have to be amended. Surveys have been conducted indicating that the costs ratio for putting right an error in the analysis phase are from 10 to 20 times lower than rectifying the same error in the evaluation phase. If the error is remedied during routine system operation, this ratio is up to 100 times higher. Evaluating the characteristics of the prototype involves asking questions like: • Does the system usually make the same decisions as an expert would? • Are the inference rules correct, consistent and complete? • Does the control strategy allow the system to consider the elements in the natural order preferred by the experts? If the system asks absurd questions in an unusual order, the users’ confidence in the system will wane. Consequently, explanations given by the system should satisfactorily describe how and why the conclusions have been reached. In addition, the test problems should cover the domain using paradigmatic cases, testing the system to the limit by setting tough cases. Another set of questions arises when evaluating the usefulness of the system, like (Mate´ & Pazos, 1988): • Does the problem solved help the user in any way? • Are the conclusions output by the system properly organised, ordered and presented in enough detail? • Is the system fast enough to satisfy the user? • Is the user interface friendly enough? The ES must be refined and verified in a laboratory


F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184

Fig. 12. LIDO Multi-Joint II System diagram.

environment before it is tested in the field. Nevertheless, new complications will appear when the target users test the system on real problems, which will take some time to correct. Field users demand high quality, speed, reliability, and ease of use and understanding of systems. Therefore, ESs need large-scale field verification before being put on the market. 3. Expert system for isokinetic evaluation Below an expert system is described for interpreting an assessment output by an isokinetics machine, which was developed according to the methodology described above. 3.1. Isokinetic assessment Specialists in the field of physical exercise and rehabilitation have been concerned with correctly assessing human muscle strength for many decades. The former aim to correctly measure muscle strength by comparing the effects obtained with a range of strength-building and fitness

programmes. Specialists in rehabilitation aim to get data about the effectiveness of therapeutic exercise on the recovery of patients convalescing from bone and muscle injuries. Athletics coaches and sports physiotherapists place the emphasis on injury prevention by identifying the underlying strength deficits and strength ratios in bilateral and reciprocal muscle groups. All these goals are based on an accurate and reliable quantification of the capacity of the human muscle to output strength. These specialists use different methods, including isometric, isotonic and isokinetic assessment, the latter of which is the most efficient and newest technique. An isokinetic exercise is the exertion of a joint across a kinetic field at a constant speed throughout (Perrin, 1988). The exercise is performed by establishing a constant speed of execution (e.g., 60⬚/s, 180⬚/s, 300⬚/s), and any additional effort employed by the individual undergoing the test will be offset by the isokinetics machine, so that the speed of the exercise remains unchanged. The intrinsic qualities of an isokinetic exercise mean that the joint under assessment is capable of the maximum performance possible within a kinetic field at a constant, previously determined speed. This makes for an efficient rehabilitation system and is ideal for quantifying the capability of a muscle group to generate torque or strength. The isokinetics machines used to measure the strengths applied in the exercises are dynamometers, which measure the performance of the joint concerned. The torque can be calculated provided the distance between the point at which the force is applied and the axis of rotation is known. Apart from the torque, they also measure the angular velocity and position of the part of the body in motion. The parameters to be considered when evaluating an isokinetic dynamometer are: the type of exercise to be performed, the range of test speeds, maximum and minimum performance or torques, and the associated software built into each dynamometer for data control and processing. The Physiotherapy School run by the Spanish National Organisation of the Blind (ONCE) has an isokinetics

Fig. 13. Fuzzy value for plausibility.

F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184


Fig. 14. Global fuzzy result of the feasibility study.

machine, based on the LIDO Multi-Joint II system (Fig. 12), which it uses to diagnose and assess joint muscle problems suffered by ONCE members. The machine is operated by a specialist physiotherapist, experienced in its use and capable of interpreting the information supplied. In order to obviate the dependency of machine use on this expert, the possibility of developing an ES was considered. This ES was to interpret the numeric and graphic information output by this machine as if it were the expert. Also, the system was to be furnished with aids and appliances that would enable its use by a blind therapist. Development started with a system that would treat knee muscles, as this was the joint troubling a greater number of patients. The system developed was called ISOCIN. 3.2. Application identification 3.2.1. Requirements plan The three main goals of ISOCIN are as follows: • Analyse isokinetic tests for the knee. The system will determine the status of the knee using the results output by the isokinetics machine. The tests will be run according to a protocol defined by the expert. • Transmit expert knowledge. There is only one available specialist at present, and it is not advisable to depend exclusively on her. Neither is there much literature on the subject, which makes it difficult to train new

specialists. The ES should offer explanations that would provide knowledge about isokinetics. • Provide a user interface adapted for the blind, as the system is to be used at the ONCE’s Physiotherapy School.

3.2.2. Application evaluation and selection All four dimensions of the feasibility study were applied to the problem, and the results were as follows (Garcı´a & Zanoletty, 1998): • Plausibility: the fuzzy value of this dimension was very high, falling between the linguistic values “very” and “absolutely”, as shown in Fig. 13. Its associated numeric value is 8.59. This indicates that ISOCIN development is possible. • Justification: the result for this dimension was almost equal to the linguistic label “absolutely” (with a numeric value of 9.15) and so the development of ISOCIN is completely justified. • Fitness: its value fell between the labels “very” and “absolutely” (with a numeric value of 8.41), showing that ISOCIN can be built using KE techniques. • Success: the result for this dimension was slightly lower, falling between “fairly” and “very” (with a numeric value of 5.4), indicating that the system could be successful, provided developers took certain risks into account.

Fig. 15. Use Case Based Model of ISOCIN.


F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184

Fig. 16. Actor Interaction Diagram for “Interpret Isokinetic Test”.

The final result of the feasibility study is a weighted arithmetic mean of the four dimensions (see Fig. 14), which falls between the labels “very” and “absolutely” (numeric value of 8.29), and thus indicates that the development of ISOCIN is highly feasible. 3.2.3. Application definition The Use Case diagram for ISOCIN (Fig. 15) shows the use cases identified for the system, which were as follows: • Perform isokinetic test: the system will communicate with LIDO in order to obtain the data for a set of exercises (called a test), following the protocol defined by the expert. These data will be stored in the system. • Interpret isokinetic test: the system interprets one stored test and provides a report on the status of the knee. • Configure the system: the ISOCIN user interface can be configured in order to facilitate its use by people with differing degrees of visual impairment. We focus here on the use case “Interpret isokinetic test”, as this is ISOCIN’s main task. This use case involves the user of the system (a physiotherapist) choosing one of the existing tests and asking the system to interpret it. There are two possible ISOCIN responses: an error message if the test was not run according to the protocol or a report with the result of the interpretation of the test and its respective explanation. The diagrams for this use case are shown in Figs. 16 and 17.

the fact that it will focus on only one of the possible isokinetic tests, defined by the following parameters: • • • • •

Joint: knee. Type of exercise: concentric/concentric. Position of the patient: seated. Movement arc: 90⬚. Protocol: as defined by the expert. According to this protocol, the patient performs several exercises at different speeds: 60, 180 and 300⬚/s.

3.3.2. Knowledge acquisition and conceptualisation Knowledge acquisition. As there is not much literature on isokinetics, the main source of knowledge for the system was the expert. The techniques used for knowledge elicitation were: • Structured and non-structured interviews. • Task analysis. • Questionnaires.

3.3.1. Solution conception The prototype developed will be a full system, except for

A total of 15 knowledge acquisition sessions were held, which are summarised in Table 2 (Garcı´a & Zanoletty, 1998). It follows from the above table that the subject requiring most sessions was the analysis of the curves associated with the exercises. The expert analysed the curve directly; however, this was out of question for the ES. The approach taken for the system was to transform the curve into a set of symbolic values and then run an analysis based on these values using a set of rules that modelled how the expert interpreted the curves (Fig. 18).

Fig. 17. Activity flow diagram for “Interpret Isokinetic Test”.

Fig. 18. Curve analysis.

3.3. Operational prototype development

F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184


Table 2 Knowledge acquisition process Session




1 2 3 4

Non-structured interview Semi-structured interview Task analysis Structured interview

Goal achieved Some questions arose Goal achieved Goal achieved

5 6

Task analysis Questionnaires


Task analysis


Structured interview



10 11

St. interview ⫹ Quest. Task analysis


Task analysis


Task analysis




Task analysis

General process Parameters for test validation Parameters for test validation Questions from preceding sessions General process confirmation Define strength peak for age, weight and sex Test analysis performed by the expert Comparison between test and mean table Define strength peak for age, weight and sex Muscular balance Curve characteristics for exercises at 60⬚/s Curve analysis for exercises at 180⬚ and 300⬚/s Curve analysis for exercises at 60⬚/s Curve analysis for exercises at 60⬚/s Curve analysis for exercises at 180⬚/s Conceptualisation. According to our proposed methodology, the first step was to develop a concept dictionary that contained the following concepts: • Test: describes the characteristics of an isokinetic test based on the chosen protocol. • Person: contains information about the patient under analysis. • Exercise60: represents the result of exercises performed at 60⬚/s. Table 3 “Test” specification

Goal achieved Goal not achieved ! 9 Goal achieved Goal achieved Goal achieved Goal achieved Goal achieved Goal not achieved (180) ! 15 Goal not achieved ! 14 Goal achieved Goal achieved

• Exercise180: represents the result of exercises performed at 180⬚/s. • Exercise300: represents the result of exercises performed at 300⬚/s. • Morphology60: contains symbolic values for the morphology of curves at 60⬚/s. One might think that concepts would also be necessary for the morphology at 180 and 300⬚/s. However, the graphical analysis of the faster exercises is simpler and does not need specific concepts. • Analysis: contains the conclusion of the system. Each concept was defined by identifying attributes, values for the attributes, and dependencies on other concepts. For

Concept name: Test Attributes



Knee, ankle, hip, elbow, shoulder, wrist Con/Con, Con/Ecc-1, Con/Ecc-2, isotonic, isometric Seated, lying


Position … Dependencies Association Assembly


Individual, analysis Exercise60, Exercise180, Exercise300 –

Fig. 19. Static conceptual model for ISOCIN.


F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184

Fig. 20. Process and control model for ISOCIN.

instance, Table 3 shows part of the specification of the concept “Test”. After all the concepts had been defined, we developed the static conceptual model or concept map for ISOCIN, which is shown in Fig. 19. An individual can perform several tests. A test is associated with one result (analysis). It also contains exercises at different speeds. Finally, an exercise at 60⬚/s is associated with its respective symbolic values (morphology). The next step was to model the strategic system knowledge to establish the process and control model. ISOCIN’s main task is to interpret an isokinetic test. This task is divided into four steps: check whether the test adheres to the protocol, calculate initial values, analyse the values (the main subtask) and allow users to add their own observations to the final report. The values are analysed in two ways: numerically (based on direct numeric values) and morphologically (based on the properties of the curve). Fig. 20 shows the complete process and control diagram. Each process and control diagram box is further analysed in order to specify how this task has to be performed. The representation used to specify tasks depends of the level of the task: • Non-leaf tasks are represented as control diagrams, showing how one task controls subtask performance.

• Leaf tasks define processes and can be represented in several ways, depending on the problem. Each ISOCIN task is represented as a set of rules, which, together, provide the solution to the problem. For instance, the task differential analysis (one of the subtasks of the numeric analysis of an isokinetic test) is described as a set of 53 rules. Table 4 shows two of these rules. The last step of the conceptualisation is synthesis, which involves the construction of the knowledge map representing the links between known attributes and inferred values. Fig. 21 shows a small part of the ISOCIN knowledge map, which deals with the determination of the result of the differential analysis (Garcı´a & Zanoletty, 1998). This knowledge map shows the following inference relationships: • The result of the differential analysis (DiffResult) depends on the analysis of extensors and flexors, the analysis of peak differences for maximal extension in exercises at 60 and 180⬚/s, the analysis of peak differences for maximal flexion in exercises at 60 and 180⬚/s, and the analysis of differences in effort for total extension and flexion at 300⬚/s. • The analysis of extensors depends on the analysis of peak

Table 4 Two of the rules for differential analysis Rule 19 Description IF THEN … Rule 36 Description


If the peak difference of maximal extensions at 180⬚/s is from 80 to 90, then the status of the left extensors is suspect Ex180.MaxExtPeakDif ⱖ 80 AND Ex180.MaxExtPeakDif ⱕ 90 Ex180.MaxExtPeakDifAn ˆ SuspectL

If the analysis of the peak difference of maximal extensions at 60⬚/s reveals a problem in the left extensors and a problem is detected (or at least a suspect result is output) by the analysis of the peak difference of maximal extensions at 180⬚/s, then the exercise at 180⬚/s confirms the propensity to problems in the left extensors Ex60.MaxExtPeakDifAn ˆ ProblemL AND (Ex180.MaxExtPeakDifAn ˆ ProblemL OR Ex180.MaxExtPeakDifAn ˆ SuspectL) Analysis.DiffResult ˆ “The exercise at 180⬚/s confirms the propensity to problems in the left extensors”

F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184


Fig. 21. Partial ISOCIN knowledge map.

differences for maximal extension in exercises at 60 and 180⬚/s. • The analysis of flexors depends on the analysis of peak differences for maximal flexion in exercises at 60 and 180⬚/s. • The analysis of each difference depends solely on the difference.

3.3.3. Formalisation The object-oriented paradigm was selected as a knowledge representation formalism for this problem, because it is better able to deal with the problem at hand. There are a series of individual points that suggest that this is one of the best formalisms for this problem. For instance, there are a set of concepts composed of attributes, whose possible values can be represented by means of classes with attributes and their access methods. There is also a group of

attributes, whose values must be calculated dynamically by a rule set. This can be done by message passing, which can emulate production rule-based operation. Finally, class inheritance can be used to simplify the representation of some of the concepts of the prototype. The acquired knowledge and the conceptual model need to be translated to the selected formalism. Fig. 22 represents the list of concepts (classes) identified in the concept base. A new class was devised to generalise all the different exercises, and these all inherit from the exercise superclass. In order to formalise the process and control model, we have to identify the different types of tasks. First, non-leaf tasks are formalised as control class methods. Second, leaf tasks are formalised as inference engine class methods, because they fire rule sets. Fig. 23 shows the new methods to be added to the control and inference engine classes. The inference engine is formalised to execute the expert

Fig. 22. Concepts of the formal model.


F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184

represent the functionality of the control class in this case. Therefore, the control class methods are turned into methods of the man/machine interface class.

Fig. 23. Control and inference engine class methods obtained from the process and control model.

knowledge represented by rules. This class has a main method in charge of starting up the reasoning process by rule chaining. The rules will be grouped by category in the other methods. These methods will send messages to the above-mentioned classes to modify their respective attributes until the reasoning process has been completed and the conclusions have been reached. The man/machine interface is represented by an MSWindows application and is an event-driven interface. This will share out the control module tasks in this ES between the man/machine interface and the inference engine classes, so there is no need for a specific class to

3.3.4. Implementation For ES implementation, we first analysed whether it was best to use a special-purpose tool, like Kappa, or a standard programming language that met a series of requirements. The balance tipped in favour of Borland Delphi because it is better suited to the proposed solution. Apart from meeting object-oriented programming requirements, Borland Delphi includes a very simple system for creating standard interfaces in the Windows environment. The standard Delphi (Paradox 5) interface has been used for database information access. The procedure for transforming the model into the final code was as follows: • The classes of the formal model are classes implemented in Delphi. • The attributes that represent different information types are converted into class attributes along with their access methods, which means that their values will be valid in all cases. • The knowledge defined as rules becomes part of a knowledge base. All the system rules are grouped together within the same unit, and the rules are divided logically according to the tasks in which they are involved. This makes it easier to locate a particular rule, if the system needs to be extended or modified. Apart from this logical

Fig. 24. ISOCIN window (Spanish) showing a left/right comparative graph.

F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184

division, the rules have been ordered to optimise chaining by the inference engine. The rule antecedent enables logical operations, whereas the consequent performs variable assignment operations and set operations for any multi-valued attributes. • The inference engine was implemented as a class, whose attributes are the objects of each class in the concept base and the methods implement the reasoning process by forward chaining the knowledge base rules, firing the rules whose antecedent is true. • As mentioned earlier, the user interface was implemented exploiting the potential of the Delphi environment. Interface design was of special importance for the entire ES, as we had to take into account that end users included individuals with severe visual impairments. Therefore, the information provided by ISOCIN is presented visually, as well as via voice synthesiser and Braille display, using the libraries and drivers of adaptive devices developed at CETTICO (Martı´n, 1996; Mun˜oz, 1997). Fig. 24 shows an ISOCIN window that depicts a comparative graph of exercises performed with the left and right legs.

4. Conclusions Knowledge-based systems and, particularly, ES development is the subject of on-going debate due primarily to two major issues. One of these issues is intrinsic to the problem type, which is characterised as follows: incomplete, often inconsistent specifications that usually address uncertain data, i.e., their real value is unknown. The other is a methodological question, as there is no proper line of development specifying the transition from the conceptual model of the problem to the formal model of the system. KE has been able to adopt some of the new techniques that have emerged in SE, such as object and event orientation, by means of which this methodological problem can be surmounted, leading to a natural derivation from the problem to the system domain. Specifically, the solution proposed here includes five major instruments: use cases to specify the ES; the concept dictionary to define and limit the content of each concept; the conceptual model to provide a static description of the system; the control and process model to model the strategic system knowledge, and the formal metamodel to describe the architecture of the ES. The proposed methodology has been applied to develop an ES (ISOCIN), which interprets the results (numeric and graphic) output by an isokinetics machine. This machine diagnoses and assesses muscle problems in the knee joint. This development project demonstrated that it was reasonable and right to use this methodological solution as opposed to conventional techniques.


Acknowledgements This paper was prepared and written with the collaboration of CETTICO (Centre of Computing and Communications Technology Transfer, Spain).

References Alonso, F., Pazos, J. (1990). Computer science engineering: integration of software engineering and knowledge engineering. In Proceedings of IAKE’90, 2nd Annual Conference of the International Association of Knowledge Engineers, San Francisco, California. Alonso, F., Maojo, V., Pazos, J. (1992). Technology transfer in computer science. In Proceedings of WFED/FMOJ Fourth International Seminar, Rio de Janeiro, Brazil. Alonso, F., Fuertes, J.L., Lo´pez, G., Montes, C. (1994). CLASER: Sistema Abierto Reutilizable para el Desarrollo y Consulta de Sistemas Expertos. In Proceedings of the Eighth International Symposium in Informatics Applications (pp. 431–438), Universidad Cato´lica del Norte, Chile. Alonso, F., Juristo, N., & Pazos, J. (1995). Trends in life-cycle models for SE and KE: proposal for a spiral-conical life-cycle approach. International Journal of Software Engineering and Knowledge Engineering, 5 (3), 445–465. Alonso, F., Juristo, N., Mate´, J. L., & Pazos, J. (1996). Software engineering and knowledge engineering: towards a common life cycle. The Journal of Systems and Software, 33, 65–79. Alonso, F., Fuertes, J.L., Martı´nez, L., Montes, C. (1997). A knowledge engineering software development methodology applied to a spiral/ conical life cycle. In Proceedings of Ninth International Conference on Software Engineering and Knowledge Engineering (SEKE’97), Madrid, Spain. Alonso, F., Barreiro, J.M., Fuertes, J.L., Martı´nez, L., Montes, C., 1999. An incremental prototyping approach to software development in knowledge engineering. In Proceedings of the Third World Multiconference on Systemics, Cybernetics and Informatics (SCI’99) and Fifth International Conference on Information System Analysis and Synthesis (ISAS’99) (pp. 76–82), Orlando, FL, USA, vol. 2. Alonso, F., Fuertes, J.L., Martı´nez, L., Montes, C. (1999). Object-oriented model for expert systems implementation. In Proceedings of Colloquium on Object Technology and System Re-engineering (to appear in book format in Ellis Horwood Publisher under Engineering Science Series), Oxford, UK. Bartholomew, J. (1988). The Times Atlas of the World. London: J. Bartholomew & Son Ltd/Times Book Ltd. Blum, B.I. (1992). The evaluation of SE. In Proceedings of 1st International Conference on Information and Knowledge Management, Baltimore, MD. Boehm, B. W. (1988). A spiral model of software development and enhancement. ACM Software Engineering Notes, 11 (4), 22–42. Garcı´a, M.M. (1993). CLASER: Conjunto de Librerı´as Abiertas para el Desarrollo y Consulta de Sistemas Expertos Reutilizable. Technical Report, Facultad de Informa´tica, Madrid. Garcı´a, C., Zanoletty, D. (1998). ISOCIN: Sistema Experto para la Interpretacio´n de Isocine´tica. MS in Knowledge Engineering thesis, Universidad Polite´cnica de Madrid, Madrid. Go´mez, A., Juristo, N., & Pazos, J. (1997). Ingenierı´a del Conocimiento: Disen˜o y Construccio´n de Sistemas Expertos. Madrid: Ceura. Martı´n, A. (1996). Desarrollo de Drivers SSIL y ESBIL, UPM-CETTICO Technical Report. Mate´, J.L., Pazos, J. (1988). Ingenierı´a del Conocimiento. Disen˜o y Construccio´n de Sistemas Expertos. Argentina: Ed. Sepa. Montes, C. (1994). MITO: Me´todo de Induccio´n Total, PhD thesis, Universidad Polite´cnica de Madrid, Madrid.


F. Alonso et al. / Expert Systems with Applications 18 (2000) 165–184

Mun˜oz, J.M. (1997). Desarrollo de Librerı´as para Manejar Adaptaciones para Invidentes, UPM-CETTICO Technical Report. Naughton, M. J. (1989). A specific knowledge acquisition methodology for expert systems development: a brief look at precision knowledge acquisition. In J. Liebowitz & D. A. De Salvo, Structuring Expert Systems. Domain, Design and Development, Englewood Cliffs, NJ: Yourdon Press.

Perrin, D.H. (1988). Isocine´tica, ejercicios y evaluacio´n, Universidad de Virginia: Ediciones Bellaterra. Spengler, D. (1918/1922). Der Untergang des Abendlandes, 8, Munich: Beck. Yourdon, E., Whitehead, K., Thomenn, J., Oppel, K., Nevermann, P. (1995). Mainstream objects. An analysis and design approach for business. New York, NY: Yourdon Press.