Using Supervised Learning Techniques for Diagnosis of Dynamic Systems

Using Supervised Learning Techniques for Diagnosis of Dynamic Systems Pedro J. Abad 1, Antonio J. Surirez', Rafael M. Gasca 2, Juan A. Ortega 2 Abstract. This paper describes an approach based on supervised diagnose systems faults are needed to maintain the systems in learning techniques for the diagnosis of dynamic systems. The levels of security, production and reliability. methodology can start with real system data or with a model of Inside the Artificial Intelligent community the dynamic systems the dynamic system. In the second case, a set of simulations of diagnosis task has been approached, in most of the cases, adapting the system is required to obtain the necessary data. In both cases, the techniques coming from the static systems diagnosis to the obtained data will be labelled according to the running conditions dynamic behaviour of the systems. This way [2] or [3] try to add of the system at the gathering data time. Label indicates the temporary information to GDE [4] running state of system: correct working or abnormal functioning On the other hand, qualitative models have also been commonly of any system component. After being labelled, data will be used for this purpose [5] [6]. treated to add additional information about the running of system. In [7] the fundaments of the based-models diagnosis, applied to The final goal is to obtain a set of decision rules by applying a the dynamic systems, are presented, and more recently [8] proposes classification tool to the set of labelled and treated data. This a consistency-based approach with qualitative models. way, any observation on the system will be classified according Other techniques, coming from the AL, have also entered in to those decision rules, having a return label indicating the the diagnosis field. Following this line, learning techniques tries to currently running state of system. Returned label will be the identify the system behaviour basing on a previous training. diagnostic. This entire learning task is carried out off-line, before Lately, some works using learning-based techniques have been the diagnosing. presented, like stochastic methods [9], neural network based learning [10] and classification systems [11]. Neural network techniques have recently been applied in diverse fields, as 1 INTRODUCTION medicine [12] or power supply [13]. Machine Learning techniques, inside the supervised learning Diagnosis determines why a system, correctly designed, doesn't field, are automated procedures based on logical operations that work like it was expected. Explanation, for this erroneous learn a task starting from a suite of examples. In the classification behaviour, represents a discrepancy with the system design. One field the attention has been centred, concretely, in approaches with diagnosis task is to determine the system elements that could cause decision trees [14], where classification is the result of a series of the erroneous behaviour according to the system observations. logical steps. These approaches are able to represent the most Monitoring process is fundamental to avoid non-real faults by complex problems if they have enough data. Applied to the small alterations in variables values. [1] Proposes a knowledge diagnosis, we can find these methods used for the classification of model for dynamic systems monitoring. temporay patterns [15] or in previous works to the current one Fault detection consists on determining, starting from the [16] [17]. system observations, when an incorrect operation of the observed The present work is centred in quantitative models. It uses system exists. When failure is detected then diagnosis will take the supervised learning techniques to obtain a rules-based model to control to find the reasons of that incorrect behaviour, diagnose dynamic systems by recognizing the correct behaviour Fault detection and diagnostic of faulty components are very models and faulty behaviour models. An approach to offer several important from the strategic point of view of the companies, due to fault causes, when there isn't an only clear cause, is presented. the economic demands and environment conservation required to Rest of the document has been organized in the following way: remain in competitive markets. This is one of the reasons causing in the next section the used methodology will be exposed and the that this is a very active investigation field. Components faults and form to carry out the diagnosis. Next a problem application process faults can cause systems damages and undesirable halt of example is described for the developed approach. To illustrate the the system. This causes the increase of costs and decrease of operation of these techniques a wide set of tests is presented. Lastly production. Therefore developing mechanisms to detect and to some improvements that are in development process are discussed. Dpto de Ingenieria Electr6nica, Sistemas Informtnticos y Automdtica. 2 PROPOSED METHODOLOGY Universidad de Huelva. E-Mail: {abadhe,[email protected]} 2 Dpto de Lenguaje y Sistemas 1nformdticos. Universidad de Sevilla. To carry out diagnosis of dynamic systems a set of decision rules E- Mail: {gasca,[email protected]} should be generated. It can be done starting from the known trajectories of the system or the simulations generated from a 2. Decision rules are generated using a supervised learning tool. model. v Relabelled trajectories * Decision rules Before starting with the methodology some concepts need to be 3. Diagnosis consists in associating an observation as defined. corresponding to behaviours family by using decision rules. Classification (observation, rules) * Diagnostic label 2.1 Definitions and notation. Definition 1: Behaviours Family. It is a finite group of 2.2 Methodology trajectories having a similar behaviour from the point of view of Proposed methodology to diagnose is an amplification of other one the diagnosis. developed in [16]. This basic methodology may present some Definition 2: Correct behaviour. It is the finite group of problems when the same system behaviours can be associated to trajectories belonging to evolutions of the system without any fault different fault reasons. In order to don't diagnose incorrectly these type. cases, in this new approach, those behaviours will be associated Definition 3: Perfect behaviour. It is the trajectory describing the with all the possible behaviours family that can cause this concrete system when all parameters take the central values of the ranges behaviour. In this way several fault causes will be offered for defined as correct. observations that can correspond to different behaviours family. Definition 4: Observation. It is a real trajectory of the dynamic Basic idea consists in obtaining a set of classification rules from system containing values of the observational variables in the a suite of system data in different behaviours modes: the correct system. behaviour and the faulty behaviours. After, those obtained Definition 5: Diagnosis. It is the identification of the observed classification rules can be used to associate an observation with behaviour of the system as belonging to a certain behaviour family model behaviour. Thus diagnosis of the observation is obtained. (diagnosis label) and according to decision rules. Process can start with real system data or with a model of the Proposed approach can be generated from two different ways: dynamic system. In the second case, a set of simulations of the "* Rules are generated starting from a group of different system is required to obtain the necessary data. In both cases, behaviour models. obtained data will be labelled according to the running conditions v Model (behaviour) * labelled trajectories of the system at the gathering data time. Label indicates the "* Rules are generated starting from a group of experimental running system state: correct working or abnormal function of any trajectories of dynamic system for the correct behaviour and system component. Final result consists in a database containing all possible fault behaviour. labelled trajectories. v Trajectories (behaviour) * labelled trajectories. Obtained database contains very similar trajectories Leaving of one of these situations the process can continue like corresponding to different behaviour family and therefore with that: different labels. To solve this problem the set of all similar 1. Similar trajectories belonging to different behaviours family are trajectories will be relabelled with new labels. This new labels will identified. These trajectories are labelled again as belonging to be composed as a mix of the older labels. Thus, relabelled both behaviours family. trajectories will be associated with anyone of the original v Similar Trajectories (different behaviour family) • behaviours family. The problem is to define when two or more relabelled trajectories, trajectories are similar. Decision taken is that several trajectories Problem Simulating Description M14ln Moe iuaig Dtbae LbligLble System Real System Observation DecisionRc ble Rules ClassiFication Labelled & Data Database Treated Treatment D M thbdasoe Evaluation DIAGNOSIS Figure 1. Proposed Methodology are similar when distance between them is lower than a magnitude. System can be modelled by the following equations, which That magnitude should be specified for each treated system. Used include a constant for each component that is used to model also distance is Euclidean distance, the faulty behaviour of the component: After being labelled and relabelled, trajectories data will be treated to add additional information about running of the system. dw This additional information will be very useful when classification dt (1) tool tries to find decision rules, because available information will be greater. This additional information should characterize the dv system further than gathering data and it is specified for each I - Controller: -- = c, (d - w.) (2) treated systems. dt A new database, which contains original trajectories plus new attributes and the corresponding label, is obtained. Sensor: w,, = c, * w (3) Final step, to obtain decision rules, is to use a classification tool with the labelled and treated database. Where T is the inertia of the motor, c., is the constant of the An aspect to highlight is that all process, until this moment, motor; c, is the constant of the controller and c, is the constant of have been development off-line, and time needed for this process is the revolution counter. not important for the diagnosis process. Component anomalous operation is caused, mainly, by the Diagnosis process consists on evaluating an observation with deviation of the component constant nominal value. These the obtained decision rules. Time spending to diagnose is only the constants stray of the considered correct values range time of evaluating obtained decision rules. Decision rules returns Some faults represent that constants take values above the the label associated to the behaviour by correspondence between correct ones and others faults represent that constants take values training data and observed data. This returned label is offered as below the correct ones. Diagnosis result should indicate, in diagnosis. addition to the faulty component, if taken values for the component Next a case study will be presented to develop this constant are below correct values or above them. methodology. Possible fault reasons that we want to identify are therefore: 'CmHigh' when values of Cm are above the correct ones; M 4 'CmLow' when values of Cm are below the correct ones; 'CsHigh' when values of Cs are above the correct ones; 'CsLow' when values of Cs are below the correct ones; 'CcHigh' when values of Cc are above the correct ones and 'CcLow' when values of Cc are below the correct ones. To describe the system correct behaviour, it is considered that values of all constants don' t have only one correct value, but rather v they can take values inside an interval that will be considered as c _ correct. This way, operation flexibility is allowed and system real behaviour is better simulated, where there is not a correct value but d rather correction margins are flexible. This produces that system doesn' t have an only correct behaviour, but rather a correct behaviours family. It represents all possible combinations of the Figure 2. The example system constants values that are inside of the defined tolerance limit. A correct behaviours family does the diagnosis more difficult, 3 CASE STUDY because it is necessary to recognize different behaviours as correct, but on the contrary it provides a more realistic vision of the system. As it has been commented previously, methodology can be used In our model the constant values considered as correct are: with real system data or with obtained data of a model simulation. In our case, the methodology will be applied to a model, which is Table I. Values for OK behaviours an idealized situation, but it offers us a clear idea of the way to act. Cm [0.98-1.02] In case of application on a real system, many difficult aspects, not Cc [0.98-1.02] mentioned here (as monitoring or small phase shift), need to be taken in account, but with the model we are only trying to present the approach. As example of dynamic system to diagnose we consider the Other considered characteristics in our system are: controller electric motor in [18] and [19]. Figure 2 represents 1. Fault is present from the beginning and it doesn' t evolve in the treated system. The motor 'M', whose rotational speed is 'w', is time. driven through a voltage 'v' by the controller 'C' which acts based 2. Behaviour change occurs instantly and starting from here it on the desired speed 'd' and the speed 'w,,' measured by the doesn t change again. revolution counter 'S'. Controller 'C' is considered as an I- controller. 3. Once the wanted angular speed has been indicated, it doesn't change until this angular speed is reached. This way, diagnosis will be carried out when the desired angular classification tool to obtain a set of decision rules, and if we have speed (d) is changed. The way to diagnose is by checking the similar trajectories with different labels then classifier can't evolution to reach the final speed. It is necessary to keep in mind correctly work; that is to say, those similar trajectories will be that in spite of existence of a failure in some component, I- incorrectly classified. Figure 7 shows an example of this. controller is able to act on the motor to reach the required final speed. Of course evolution of the system to reach the desired final speed will be different. This difference in the behaviour will allow 20 the diagnosis. T 10 / % f • - VW INTEG(F2) Wr = INTE G(/ Cm Wm'•--- -- cs F2 = Cc*(d-W.g) F2 / F = (•.y-YV/T Cc 0 6 12 18 24 30 Time (Second) Figure 4. OK Behaviour P2O Figure 3. Forrester diagram First step, therefore, is performing system simulations in 15 different behaviours modes. In our case, system has been modelled as a Forrester diagram [20], to be able to simulate using the 7 - simulation tool VEMSIM&. Forrester diagram generated for the 10\ .... - -j -. system is presented in figure 3. J Simulated behaviours will be those that we want to diagnose. / They will be: OK for correct behaviour and CmHigh, CmLow, CsHigh, CsLow, CcHigh, CcLow for each component fault above mentioned. 0 A behaviour family will represent each one of these behaviours. 0 6 12 1i 24 30 Simulations values are shown in table 2. Time (Second) Table 2. System values for simulation Figure 5. CmHigh Behaviour T 3 D 10 20 W 5 Time Step 0.1 15 For the correct behaviour the constant values are into [0.98- 10 ------- -- 1.02]. Values to simulate behaviours above the correct one are into -J [1.02-5]. Values to simulate behaviours bellow the correct one are into [0-0.98]. 5 7 _ Constants values for simulated behaviours have been elected by random with the Monte Carlo method following a uniform 0 distribution. Number of simulations per behaviour will be 100. 0 6 12 18 24 30 Label corresponding to behaviour is placed to each one of the Time (Second) trajectories. This way, a database containing 700 labelled trajectories is obtained. Figure 6. CcLow Behaviour Trajectories are composed with values of the variable 'w,,' in each time step. Reason to select variable 'w,,' and not 'w' is that 'w,,,' is the only observable variable in the real system. To solve this problem a new label will be assigned to very In figures 4, 5 and 6 different system behaviours are shown, similar trajectories. A mixture of labels of all similar trajectories Obtained database has similar trajectories belong to different will compose the new label. This way, next step is to find all behaviours. This way several very similar trajectories have similar trajectories into the database and assigning a new label. different labels. This is a problem, because our final goal is to use a It is necessary to define when two or more trajectories are .Max speed time (MST). It is the moment in which the highest similar. Two trajectories are considered similar when distance revolution speed is reached. between them is smaller than a magnitude. Distance between This way a new database containing trajectories plus new trajectories is measured as Euclidean Distance and magnitude attributes is generated. chosen is 10% of the Euclidean distance between the two further Data in new database have the following form: away trajectories for the correct behaviour. This magnitude in our RT, SS, MS, MST, Win[1], DP[f1, 111], ....... Win[n], DP[n], I[n], example is 0.45. LABEL Final step is performing supervised learning with the obtained database. Classification tool selected to perform the supervised 20 learning is C4.5 [21]. What is gotten with this tool is to characterize each one of the behaviour families according to the 15--values of the attributes that have been provided. Result is a decision tree and an equivalent set of decision rules. These rules / will be the way to do the diagnosis. In our example classifier 10 ------ - -- obtains 27 rules with an error rate of 1.2%. This mean that 1.2% of ýT "trajectories are not correctly classified with those rules. 3.1 Diagnosis 0 The way to do the diagnosis is evaluate the observed data with the 0 6 12 18 24 0 obtained rules. Because in rules appear attributes that have been calculated and Time (Second) not appear in observed data, same attributes should be calculated for observed data in order to be able to classify with those rules. Figure 7. Behaviour CcHigh vs CmHigh This way in the moment that one observed data is gathered all possible attributes should be calculated. After that, decision rules After this process we obtain a new database with all similar are evaluated with two possible results: a label is returned or trajectories re-labelled as corresponding with all behaviours of the information is insufficient to evaluate all rules. In the first case the similar trajectories. returned label is the result of the diagnosis. In the second one we Next step is to calculate new attributes of each trajectory with need to wait more information in further moments. the goal that classifier has more information to generate decision If we want to diagnose the system with another running rules. These new attributes must be representative for each conditions, we should have prepared the decision rules set for those trajectory. specific conditions. I. e. if we want to diagnose this system when For each trajectory point next attributes have been calculated: current rotational speed is 12 rad/sec and desired rotational speed " Distance to perfect behaviour. It indicates how far away is is 7 rad/sec, we should have generated a set of decision rules for current trajectory from perfect behaviour (above defined). It those conditions and we will use them in the diagnosis moment. is calculated as: DP(i) = Wm[i]- Wmpf[i] (4) 4 RESULTS ON THE EXAMPLE SYSTEM To evaluate the proposed methodology a set of tests have been Where Wm[i] is the treated point in the current trajectory and done. Wmpj[i] is the correspondent point in the perfect behaviour. Observational data have been obtained by simulating the system " Integral. It is the magnitude returned by numerical integration with specific conditions for the test. This way a test trajectory is between current point and the precedent one. It represents the obtained and the diagnosis correct result is known, because it must closed area between them. It is calculated by approximating be the corresponding to the simulated conditions. as follow: Conditions of the test are the same above mentioned. We remember them in table 3: p[ijjpi~i-1](5 l(i) = T's x _____ 5 2 Table 3. Tests conditions T 3 Where Ts is the time step in the simulation, p[i] is the current D 10 treated point and p[i-1] the precedent one. W initial 5 In addition next attributes will be calculated for each trajectory: Time Step 0.1 "* Rise Time (RT). It is the moment in which desired revolution Values for OK [0.98 - 1.02] speed is reached for first time. Values for HIGH [1.02 -5] "* Steady state (SS). It is the moment in which desired Values for LOW [0 - 0.98] revolution speed is reached definitively. "• Max speed (MS). It is the value of the highest revolution In table 4 we can see results for the tests: reached speed. Table 4. Tests results times, methodology returns an incorrect diagnosis, but in general VALUE OF THE DIAGNOSIS DIAGNOSIS offered results are acceptable. CORRECT WITH SIMPLE WITH This occurs because there are very similar trajectories belonging Cm Cc Cs DIAG S LABELLED LEL to different behaviours, and classifier cannot correctly select the LABELLED rules to difference them. 1 1 1.03 CS HIGH CS HIGH CS HIGH To solve this problem the new methodology proposes the re- 1 1 1.07 CS HIGH CS HIGH CS HIGH labelled of all similar trajectories as have been above mentioned. I 1 1.1 CS HIGH CS HIGH CS HIGH Obtained results show that the new methodology offers a multiple I 1 1.5 CS HIGH CS HIGH CS HIGH diagnosis when the previous one can't find the correct fault. I 1 3 CS HIGH CS HIGH CS HIGH Among the multiple offered diagnoses, near to all tests return the 1 1.03 1 CCGH CS OK OK correct one. CC HIGH C It is important to highlight that, in tests where behaviour is far 1 1.07 1 CC HIGH CM HIGH CM HIGH of the correct one, offered diagnosis is the correct one. CC HIGH In the set of presented tests the diagnosis is correct in 58.33 % CM HIGH of the cases. Correct diagnosis is offered, among others, in 30.55 % 1 1.5 1 CC HIGH CC HIGH CC HIGH of the cases. An incorrect diagnosis is offered in 2.7 % of the cases. 1 2 1 CC HIGH CC HIGH CC HIGH The fault is not detected in 8.33 % of the cases. Otherwise, never 1 3 1 CC HIGH CC HIGH CC HIGH detect failure when failure doesn't exist. 1.03 1 1 CM HIGH OK OKCI CS LOW 1.07 1 1 CM HIGH CM HIGH CC HIGH 5 CONCLUSIONS AND FURTHER CM HIGH CC HIGHI O K 1.1 1 1 CM HIGH CM HIGH CCHGWO K CM HIGH Presented methodology is able to perform diagnosis of dynamic 1.5 1 1 CM HIGH CM HIGH CM HIGH 2 1 1 CM HIGH CM HIGH CM HIGH systems and it is independent of the system type. In fact, one of 3_ 1 1 CM HIGH CM HIGH CM HIGH further works is to apply this methodology to a non-linear dynamic 3 1 1 CM HIGH CM HIGH CM HIGHsytm 1 1 .97 S LO OKCS LOW j system. 1 1 0.97 CS LOW OK OK This capacity is due to the fact that the methodology is only 1 1 0.93 CS LOW CS LOW CS LOW centred in the evolution characteristics of the system for the correct 1 1 0.89 CS LOW CS LOW CS LOW behaviour or faulty behaviours. 1 1 0.85 CS LOW CS LOW CS LOW Another characteristic of the methodology is that the diagnosis 1 1 0.5 CS LOW CS LOW CS LOW can be performed in a very simple way, and a very little 1 1 0.1 CS LOW CS LOW CS LOW computational time is required. 1 0.97 1 CC LOW OK OK Certain systems, as the presented in the example, can produce 1 0.93 1 CC LOW CC LOW CC LOW j similar behaviours for different fault reasons. This is due to CM LOW relationship among variables that govern the system behaviour. 1 0.89 1 CC LOW CC LOW CC LOW j This relationship, among system variables, can produce that an 1 0.89 1 CCLM CCLW I_ CM LOW alteration of a variable would be compensated by the alteration of 1 0.85 1 CC LOW CC LOW CC LOW another variable in contrary sense. To solve this problem, ____ CM LOW 1 0.5 1 CC LOW CC LOW CC LOW methodology assigns multiple fault reasons to system behaviours 1 0.1 1 CC LOW CC LOW CC LOW that could be produced by different fault reasons. This way a 0.97 1 1 CM LOW OK OK multiple diagnosis is offered in those situations. 0.93 1 1 CMLOW CCLow CC LOW j Another further work is to be able to diagnose dynamic system CM LOW when multiple fault occurs at the same time, is to say, identifying CM LOW CM LOW CC LOW j system behaviours when more than one component is faulty. 0.89 1 CMC LOLOWLO CM LOW CC LOW 0.85 1 1 CM LOW CM LOW CM LOW CM LOW ACKNOWLEDGMENTS 0.5 1 1 CM LOW CM LOW CM LOW 0.1 1 1 CM LOW CM LOW CM LOW This work has been partiality financed by the Comisi6n 0.99 0.98 1.02 OK OK OK Interministerial de Ciencia y Tecnologia (DP12000-0666-C02-02) 1 1.02 1.02 OK OK OK and the Modelizaci6n Matemditica Redes y Multimedia 0.98 1 0.98 OK OK OK investigation group of the University of Huelva. 0.98 1.02 1.02 OK OK OK 0.99 1.01 1.01 OK OK OK 1.01 1 0.99 OK OK OK REFERENCES We can see that diagnosis methodology with simple labelled doen'toffr acorec dignoticin est tat re eryearof he [1] C. J. Alonso, J. A. Maestro, J. B. Pulido y C. Llamas. doesn't offer a correct diagnostic in tests that are very near of the Monitorizaci6n de Sisteinas Dinmicos: hacia una Caracterizaci6n correct behaviour. In those cases the fault is not detected. Other en el Nivel de Conocimiento. In proceedings of the I Jomadas de Trabajo sobre Diagnosis. Valladolid 2001. [2] W. Hamscher. Diagnosis devices with hierarchic structure and known component jailure models. In proceedings of the 6th Conference on Al Applications.. 1990 [3] Dague, P y otros. When Oscillators stop oscillating. In proceedings of IJCAI-91 [4] J. De Kleer y B. Williams. Diagnosing multiple faults. Artificial Intelligence 32, 97-130, 1987 [5] K. Bousson, y L. Trave-Massuyes A computational causal model for process supervision. Technical Report 92147, LAAS-CNRS, Toulouse, France. 1992 [6] P. Mosterman llybrid dynamic systems: a hybrid bond graph modeling paradigm and its applications in diagnosis. Tesis Doctoral Vanderbilt University, Nashville, Tennessee, USA. 1997. [7] P. Struss. Fundamentals of model-based diagnosis of dynamic systems. Proc. IJCAI'97. 1997. [8] B. Pulido, C. Alonso y F. Acebes. Consistency-based Diagnosis of Dynamics Systems using quantitative models and off-line dependency-recording. In proceedings of DX-01.2001 [9] A.D. Pouliezos y G.S. Stavrakakis. Real time fault monitoring of industrial process. Microprocessor-based systems engineering. Kluwer Academic Publishers, Dordrecht, 1994. [10] V. Venkatusubramanian and K. Chan. A neural network methodology jbr process jault diagnosis. Journal of Artificial Intelligence in Chemical Engineering, 35:1993-2001. 1995 [11] S. Leonhart y M. Ayoubi. Methods offiault diagnosis. Control Engineering Practice, 5. 1997. [12] A. Sim6n, L. Alonso y A. Ant6n. Sistema hibrido borroso para ayuda del diagndstico del glaucoma. In proceedings of the I Jornadas de Trabajo sobre Diagnosis. Valladolid 2001. [13] S. Saludes, A. Vargas y J. R. Perin. Aplicacidn de la red neuronal SOM para la deteccidn de fiallos desconocidos en un grtpo hidroeldctrico. In proceedings of the I Jomadas de Trabajo sobre Diagnosis. Valladolid 2001. [14] J. Ross Quinlan. Induction of decision trees. Machine learning, 1986 [15] Juan J. Rodriguez, Carlos J. Alonso y Q. Isaac Moro. Clasificaci6n de patrones temporales en sistemas dinimicos mediante Boosting y Alineamiento dinamico temporal. In proceedings of the I Jomadas de Trabajo sobre Diagnosis. Valladolid 2001. [16] Pedro J. Abad y Antonio J. Suarez. Diagnosis de Sistemas Dindmicos Basada en Aprendizaje Supervisado Qff-line. In proceedings of the I Jomadas de Trabajo sobre Diagnosis. Valladolid 2001. [17] Antonio J. Suarez y Pedro J. Abad. Aplicaci6n de Thcnicas de Aprendizqje a la diagnosis de Sistemas Dinumicos con Etiquetado Mtltiple. In proceedings of the IX CAEPIA -TTIA. 2001. [18] A. Panati and D. T. Drup6 Stated based vs simulation-based diagnosis of dynamic system. ECA12000. !4Th European Conference on Artificial Intelligent. 2000. [19] A. Malik and P. Struss. Diagnosis of Dynamic Systems does not necessarily require simulation. Workshop Notes of the Seventh International Workshop on Principles of Diagnosis DX-96 Montreal. 1996. [20] J. W. Forrester Principles of systems. Wright-Allen Press 1968. [21] J. Ross Quinlan. C45:Program for Machine Learning. Morgan Kaufman, 1993.