Integrating PCA and structural model decomposition to improve fault monitoring and diagnosis with varying operation points

Engineering Applications of Artificial Intelligence 122 (2023) 106145 Contents lists available at ScienceDirect Engineering Applications of Artificial Intelligence journal homepage: www.elsevier.com/locate/engappai Integrating PCA and structural model decomposition to improve fault monitoring and diagnosis with varying operation points D. Garcia-Alvarez∗,1, A. Bregon1, B. Pulido1, C.J. Alonso-Gonzalez1 Departamento de Informática, University of Valladolid, Valladolid, 47011, Spain ARTICLE INFO Keywords: Fault diagnosis Model decomposition Possible Conflicts Multivariate Statistical Process Control Principal Component Analysis Multiple Operation Points Systems ABSTRACT Fast and efficient fault monitoring and diagnostics methods are essential for fault diagnosis and prognosis tasks in Health Monitoring Systems. These tasks are even more complicated when facing dynamic systems with multiple operation points. This article introduces a symbiotic solution for fault detection and isolation, based on the integration of two complementary techniques: Possible Conflicts (PCs), a model-based diagnosis technique from the Artificial Intelligence (AI) community, and Principal Component Analysis (PCA), a Multivariate Statistical Process Control (MSPC) technique. Our proposal improves the PCA-based fault detection in systems with multiple operation points and transient states and provides a straightforward fault isolation stage for PCA. At the same time, the proposal increases the robustness for fault detection using PCs through the application of PCA to the residual signals. PCA has the ability to filter out residual deviations caused by model uncertainties that can lead to a high number of false positives. The proposed method has been successfully tested in a real-world plant with accurate fault detection results. The plant has noisy sensors and a system model without the same accuracy at each operation point and transient states. 1. Introduction Complex dynamic systems, such as industrial and aerospace systems, require efficient fault diagnosis solutions to ease the prognosis and health monitoring tasks. Fault monitoring methods based on process data history have proven to be successful techniques for this job by directly collecting and analyzing the system’s measurements (Venkatasubramanian et al.,2003). However, accurate and quick real-time monitoring using these techniques can be compromised when large amounts of highly correlated data are being managed. One widely used technique to overcome such a flaw is PCA (Jackson and Mudholkar, 1979;Wold et al.,1987). PCA is a MSPC technique that has been used as a monitoring approach in many industrial processes (Kourti and MacGregor,1996; Shlens,2005;Ferrer,2007). PCA is capable of finding the principal sources of variability in the space of the measured variables, thus allowing the dimensionality of the original space to be reduced in order to fit into a new one with the minimum number of uncorrelated variables required to explain the process trends (known as latent variables or components) (Camacho et al.,2009;Banguero et al.,2020). This reduction is carried out by searching linear combinations between the measured variables, and the transformation matrix procedure used to compute the components is known as the PCA model. ∗Corresponding author. E-mail addresses: [email protected] (D. Garcia-Alvarez), [email protected] (A. Bregon), [email protected] (B. Pulido), [email protected] (C.J. Alonso-Gonzalez). 1This work has been partially supported by the Spanish Ministerio de Ciencia e Innovación under grant PID2021-126659OB-I00. Nevertheless, PCA presents important limitations when dealing with continuous processes that go through different operation points. Changes in the current operation point can be caused by variations in product specifications, input variables or set-points of the system. In all these situations, the relationship between process variables will change, leading to wrong fault detections caused by discrepancies between the trends captured by the PCA model and the new process state. As pointed out by Hwang and Han (1999) and Tien et al. (2004), several solutions have been proposed to overcome such issues, such as building a PCA model for each operation point (Zhao et al.,2004,2006), updating the model to reflect the changes, or developing a conventional PCA model considering all the operation points. However, these solutions could face difficulties when dealing with a large number of operation points due to the necessity of off-line PCA model fitting, transient monitoring, or online model reformulation, among others (Lane et al., 2003;Zarzo and Ferrer,2004;De Ketelaere et al.,2015;Portnoy et al.,2016;Rafferty et al.,2016;Bakdi and Kouadri,2017;Rezamand et al.,2020). Additionally, some of these solutions might exhibit poor detectability performance due to the difficulty for the PCA model to distinguish between the fault occurrence and the change in the operation point (García-Álvarez et al.,2012). Finally, PCA provides little support for fault isolation (Gertler et al.,1999). https://doi.org/10.1016/j.engappai.2023.106145 Received 27 April 2022; Received in revised form 12 January 2023; Accepted 10 March 2023 Available online 22 March 2023 0952-1976/©2023 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). D. Garcia-Alvarez, A. Bregon, B. Pulido et al. Engineering Applications of Artificial Intelligence 122 (2023) 106145 On the other hand, online fault diagnosis approaches based on analytical models (Blanke et al.,2006;Gertler,1998) require quick and robust detection methods to determine significant deviations between observed and expected behavior. These deviations, known as residuals, are related to analytical redundancy derived from the system model. The structure of these residuals can be computed off-line. However, the current value of the residuals are computed on-line. Whenever the value of a residual exceeds a given threshold, a fault detection is performed and the set of constraints used to derive the analytical redundancy expression is considered to be non-consistent with the observations. After this process, fault isolation is straightforward, and a reduced set of faulty candidates can be easily computed. Residuals can be computed using different methods, such as parityequations, state-observers, or parameter estimation (Blanke et al.,2006; Gertler et al.,1999). In this work, we focus on Analytical Redundancy Relations (ARRs) (Blanke et al.,2006;Cordier et al.,2004) obtained through structural analysis. More precisely, our work uses PCs (Pulido and Alonso-González,2004). The computation of the set of PCs is a model decomposition technique from the AI approach to modelbased diagnosis. The set of PCs identify the whole set of subsystems with minimal analytical redundancy, similar to ARRs or Minimally Structural Overdetermined Sets (MSO) (Krysander et al.,2008). Each PC has a model that can be used to generate a residual. These residuals are used in a consistency-based diagnosis (CBD) approach and are able to automatically perform fault detection, isolation and identification (Bregon et al.,2012;Moya Alonso et al.,2013). CBD is the most used approach to model-based diagnosis within the AI community. CBD with PCs, like any other model-based method based on analytical redundancy, requires detailed models of the process to compute residuals. In the absence of faults, residuals are theoretically zero. However, this is not usually the case. Even in simple processes, uncertainty in the models and noise in the sensors require the implementation of some kind of state observer to track the process, thus increasing the complexity of the fault diagnosis system. Still, residuals are usually noisy and some statistical decision-making process, for instance the z-test, is needed to assess a significant deviation of the residuals, which usually implies a compromise between false positive and false negative detections, plus some delay in the fault detection process. Additionally, the presence of nonlinearities poses a new set of problems. Each different operation point may require a different set of models or, at least, a new set of model and observer parameters. In both cases, it is necessary to design a bank of observers for each operation point of interest, which is a non-trivial task for non-linear systems and usually demands several properties from the system models. A related problem is tracking the system in transitory states, which usually generates false positive detections. An alternative approach is to use the same models in different operation points, avoiding the implementation of state observers, to keep the diagnosis system as simple as possible (Moya Alonso et al., 2013;Pulido et al.,2015). Clearly, the residuals will be noisier and may show some bias, with non-zero means even in the absence of faults in some operation points. Our hypothesis is, that in the absence of faults, the correlations between the residuals will essentially be constant in each operation point, while the onset of a fault will modify the mentioned correlations, because each residual is only sensitive to an (ideally small) subset of faults. Hence fault detection will not be based on residual deviations from zero, but on the alteration of the residual’s correlation structure. These alterations can be detected by PCA statistical process monitoring. Summarizing, in this work, we propose to combine PCA together with CBD with Possible Conflicts to improve the overall diagnosis process for systems with multiple operation points. PC models can track the system state in different operation points (Moya Alonso et al.,2013; Pulido et al.,2015;Bregon et al.,2016). Our proposal is to use PCA to analyze the trends among residual signals, instead of using direct measurements from the system. The presence of faults will modify the correlations between residuals, leading to a fault detection. After this step, contribution analysis is used to determine those variables, i.e., residuals, responsible for such deviations. Deviated residuals will then be used to compute the set of faulty candidates using CBD. The proposed combination of techniques has been tested in a realworld laboratory plant consisting of two connected tanks with liquid input and output and level control. This system presents two main weaknesses, the signals collected by sensors are significantly noisy and the fitted model does not represent the model with the same accuracy at each operation point. Our solution has proven to be successful in accurately detecting faults at different operation points and in designing a complete and straightforward fault detection and isolation solution for PCA. In addition, it has been observed that the deviations of the residuals due to lack of model accuracy were captured by the PCA model, leading to the reduction of false positives in the detection phase. The summary of the main contributions of this proposal are: first, using CBD, which requires only correct behavior models, we can detect and isolate any type of failure, without imposing simplifying or unrealistic assumptions about fault models. Second, the method does not require faulty behavioral data, because PCA models can be effectively trained with correct behavioral data, and it can be used to perform robust fault detection by just analyzing the residual signals; moreover, the PCA model can be trained with a relatively small amount of data. Third, we are able to perform accurate and timely fault detection and isolation in different operation points, without the need of having dedicated or complex PCA models for each operation point. Last, but not least, for the case study the proposal is able to track the system during transients among different operating point regions without detecting false alarms using relatively simple analytical and PCA models. The rest of the article is structured as follows. Sections 2and 3 briefly present the theoretical background of PCA and PCs, respectively, as well as their use as fault detection and diagnosis techniques. Section 4introduces the real-world case study used to apply the integration proposal. Section 5describes the new approach where both techniques are integrated. Section 6shows the experimental results obtained for the case study. Finally, Section 7discusses the results of the approach and outlines its main conclusions. 2. Principal component analysis monitoring The PCA model is fitted by using data in Normal Operation Conditions (NOC) arranged into a data matrix X∈R𝐾×𝐽, where 𝐾is the number of time observations and 𝐽the number of measured variables in the process. The columns of matrix Xare called variables (x𝑗) and represent the values of every variable along time, having these variables (columns) means 𝑥1, 𝑥2,…, 𝑥𝐽and variances 𝑠1, 𝑠2,…, 𝑠𝐽. The rows are called individuals or observations (z𝑇 𝑘). It is advisable to normalize every variable to zero mean and unit variance. PCA can be computed using the decomposition of the covariance matrix (S=1 𝐾−1 X𝑇X) and performing the Singular Value Decomposition (SVD) over it: S=VΛV𝑇 where Λ∈R𝐽×𝐽is a diagonal matrix. Λcontains the eigenvalues (𝜆) of Sin its diagonal sorted in decreasing order (𝜆1≥𝜆2≥⋯≥𝜆𝑟𝑎𝑛𝑘(X)≥0). Choosing 𝐴eigenvectors of Vcorresponding to the greatest 𝐴 eigenvalues, matrix P1∶𝐴∈R𝐽×𝐴is set. It transforms the space of the measured variables into the reduced dimension space of the latent variables or principal components: T=XP1∶𝐴 Columns in P1∶𝐴are called loadings p𝑎and T∈R𝐾×𝐴is the score matrix whose rows τ𝑇 𝑎= [𝜏1, 𝜏2,…, 𝜏𝐴]𝑇∈R1×𝐴are scores in the 𝑎th observation and columns the principal components t𝑎. Scores can be transformed into the original measured variable space  X=TP𝑇 1∶𝐴 and the residual matrix Ecan be set as the differences between the 2 D. Garcia-Alvarez, A. Bregon, B. Pulido et al. Engineering Applications of Artificial Intelligence 122 (2023) 106145 original variables and the reconstructed variables E=X− X. The PCA computation can also be performed by using the Non-linear Iterative Partial Least Squares (NIPALS) algorithm (Wold et al.,1987). There are several methods proposed for selecting the number of principal components (Jackson,2003;Weighell et al.,2001;Chiang et al.,2001). However, cross validation is considered one of the best ways to choose the number of principal components (Eastment and Krzanowski,1982;Camacho,2007;Bro et al.,2008;Camacho et al., 2010). 2.1. PCA as a fault detection technique From the point of view of fault detection, two statistics are used to establish control charts to monitor processes (Chiang et al.,2001): •Hotelling’s statistic (𝑇2): for a new process observation z𝑇∈ R1×𝐽, this statistic can be calculated as the sum of the squared 𝐴principal scores of the observation divided by the related eigenvalue: 𝑇2= 𝐴 ∑ 𝑖=1 𝜏2 𝑖 𝜆𝑖 Using this statistic, the process is considered normal for a given significance level 𝛼if: 𝑇2≤𝑇2 𝛼=(𝐾2− 1)𝐴 𝐾(𝐾−𝐴)𝐹𝛼(𝐴, 𝐾 −𝐴) where 𝐹𝛼(𝐴, 𝐾 −𝐴)is the critical value (100(1 − 𝛼)% percentile) of the Fisher–Snedecor distribution with 𝐴and 𝐾−𝐴degrees of freedom where 𝛼is the level of significance which usually takes values between 5% and 1%. •Squared prediction error (SPE) or 𝑄statistic: this statistic is a measurement of goodness of fit of the sample into the model and can be calculated for a normalized observation z𝑇: 𝑄=r𝑇r with r= (I−P1∶𝐴P𝑇 1∶𝐴)z, where ris the residual between the observation and its projection onto the reduced dimension space. In this case, the upper limit of this statistic can be set: 𝑄𝛼=𝜃1[ℎ0𝑐𝛼√2𝜃2 𝜃1 + 1 + 𝜃2ℎ0(ℎ0− 1) 𝜃2 1]1 ℎ0 where: 𝜃𝑖= 𝐽 ∑ 𝑗=𝐴+1 𝜆𝑖 𝑗ℎ0= 1 − 2𝜃1𝜃3 3𝜃2 2 𝑐𝛼being the 100(1 − 𝛼)standardized normal percentile and 𝜆𝑖 𝑗the eigenvalues of the residual covariance matrix E𝑇E∕(𝐾− 1). Although PCA does not provide much information to develop a fault isolation approach, Contribution Analysis (Kourti and MacGregor, 1996;Ferrer,2007) is a technique which has been proposed as a first attempt for fault isolation. It computes the influence of each variable to the computed value of 𝑄and 𝑇2statistics. These contributions (𝑐𝑜𝑛𝑡) can be plotted in a bar plot, which is shown together with lines marking the standard deviation and three times the standard deviation in order to detect abnormal values of the contributions. •Bar plots of normalized errors of the variables. When an observation z𝑇= [𝑧1, 𝑧2,…, 𝑧𝐽]𝑇∈R1×𝐽falls outside the limits in the 𝑄statistic, the normalized error for each value 𝑧𝑗of z𝑇is calculated as: 𝑐𝑜𝑛𝑡𝑧𝑗= (𝑧𝑗−𝑥𝑗)∕𝑠𝑗 •Bar plots of normalized scores. If the 𝑇2statistic value falls outside the threshold set, the normalized scores can be plotted for the scores of a given observation outside the limits τ𝑇as: 𝑐𝑜𝑛𝑡𝜏𝑎=𝜏𝑎∕𝜆𝑎 •Variable contributions to individual scores. Bar plots of normalized scores do not provide information about the variables related to a fault. This contribution analysis allows us to plot the contribution of each observation 𝑧𝑗to the computation of the 𝑎th score 𝜏𝑎(normally, those with a high normalized value): 𝑐𝑜𝑛𝑡𝑧𝑗,𝜏𝑎=𝑧𝑗𝑝𝑗,𝑎 where 𝑝𝑗,𝑎 are elements of P1∶𝐴 •Overall Average variable contributions. Because it is very common that more than one score has a high value when a fault is detected, it is very useful to compute the overall average contribution per variable, instead of drawing a bar plot for every score with a high value (Kourti and MacGregor,1996). 2.2. Limitations of PCA as a fault detection technique PCA-based fault detection and process monitoring techniques are widely used in academia and industry. The reason for its success is due to several factors. In general, PCA-based monitoring techniques only require NOC data collected from the process, considering that the amount of data must be large enough to capture the principal trends of the process. In addition, the PCA model and the process monitoring statistic computation do not have a high computational cost. Furthermore, despite the fact that PCA-based monitoring techniques do not provide a complete fault isolation procedure, the contribution analysis can be used to identify the deviating variables related to monitoring plot triggers. However, PCA-based monitoring approaches also present some drawbacks, particularly when the process has several operation points (Hwang and Han,1999;Tien et al.,2004). This circumstance makes it necessary to consider modifications to the classical PCA approach, or even to discard the use of this technique when the number of operation points is high. Another drawback is directly related to the monitoring of transient states that some processes present when switching from one operation point to another. Transient state monitoring requires specific PCA-based techniques that are not as simple as classical PCA approaches (García-Álvarez et al.,2012). The main target of this work is to improve PCA as a complete fault detection and isolation technique in order to give it the capability of isolating the possible source of a detected fault and also to improve the detection process, considering the mentioned drawbacks when a process presents several operation points and transient states. This improvement is done by using a model-based technique that we present next. 3. Model-based diagnosis using Possible Conflicts As mentioned in Section 1, the set of Possible Conflicts (Pulido and Alonso-González,2004) is a model decomposition method from the model-based AI community (usually known as DX community) (Reiter, 1987;Hamscher et al.,1992) that can find off-line the complete set of subsystems with minimal analytical redundancy. PCs can be used in a model-based approach to perform FDII (Fault Detection, Isolation and Identification). Additionally, we have shown (Armengol et al.,2009; Pulido and Alonso-González,2004) that PCs are, under given assumptions, equivalent to the structure of the set of minimal ARRs (Cordier et al.,2004) and MSO sets of equations (Krysander et al.,2008), all used in structural approaches to FDII in the Control Engineering community (usually known as FDI2community). 2FDI stands for Fault Detection and Isolation. 3 D. Garcia-Alvarez, A. Bregon, B. Pulido et al. Engineering Applications of Artificial Intelligence 122 (2023) 106145 The main idea of the DX approach is to link each equation (or component) in the model with a correctness assumption and to find differences between the measured variables and the model estimations. Whenever a difference is found between these two signals (a residual in FDI terminology) the fault isolation process searches for those minimal sets of equations used to estimate the conflicting value (a discrepancy in DX terminology), which are named minimal conflicts (Cordier et al., 2004) (formally, conflicts are the set of correctness assumptions linked to those sets of equations leading to a discrepancy (Hamscher et al., 1992)). The computation of the set of PCs (Pulido and Alonso-González, 2004) is performed completely off-line in three steps: first, to obtain an abstract representation of the system as an hypergraph; then to search for the whole set of minimal over-determined subsets of equations; and finally to check if these sets of equations can be solved using local propagation alone.3These three stages are explained as follows: 1. Generating an abstract representation of the system model: The computation of the set of PCs needs an abstract representation of the system: an hypergraph (Murota,2012). This type of representation only requires information about the constraints or equations in the model, and their relationship with the known and unknown variables in those equations. Thus, the hypergraph provides a representation of the set of equations from a structural point of view, i.e., it provides a structural model of the system. 2. Searching for the set of Minimal Evaluation Chains (MECs): In this step, a complete search algorithm finds the whole set of minimally overdetermined subsets (subhypergraphs) in the set of equations. Each subset is called a Minimal Evaluation Chain and represents a set of equations with exactly one more equation than unknown variables. But MECs do not have information about how the set of equations can be solved, if they can be solved at all, because they have no causality information. 3. Searching for the set of Minimal Evaluation Models (MEMs): this step works at the behavioral level in the model, by introducing causal information. In this stage, we consider all available information about causality for each constraint in a MEC. When a variable within a constraint can be solved assuming the rest of the variables are known, this is called an interpretation, i.e. a feasible causal assignment. For example, for the equation 𝑎= 𝑏×𝑐, there are three possible interpretations: 𝑎∶= 𝑏×𝑐,𝑏∶= 𝑎∕𝑐, and 𝑐∶= 𝑎∕𝑏, if both 𝑏and 𝑐are different from 0. This stage searches for all the causally consistent interpretations for each constraint in a MEC, which is called the Minimal Evaluation Model. Each MEM represents a globally consistent causal assignment within a MEC and can be used to estimate the behavior of a part of the system. Each MEC can lead to none, one or multiple MEMs. One MEC can have no MEM if the system has no valid computational assignment (for instance, computing the flow in a pipe given pressure differences is feasible, but using the flows to estimate the pressure difference between both sides is almost impossible in real world applications). A Possible Conflict is a MEC which has at least one MEM. The results of the third stage provide not only the set of equations for each PC, but also a graphical model (directed hypergraph) which is the computational way to solve the set of equations (traversing the directed hypergraph from leaves to 3In the context of Consistency-based diagnosis, local propagation means solving one equation in one unknown in each computation step. This is imposed in order to be able to trace back which equations or component models were used to estimate a variable leading to a residual activation. If this is not possible, and several equations must be solved simultaneously, the set of equations is considered as one super-constraint, with its own correctness assumption, so the reasoning process is the same as before. Fig. 1. Two tank system. the root, which is the so-called discrepancy node, and also the variable used to compute the residual as the difference between their observed and estimated values). Software tools are available to compute the set of PCs4(Pulido et al.,2016) given a system model expressed as a set of Differential Algebraic Equations (DAEs), though other software tools do exist in the model-based diagnosis community to build similar executable models for different types of equations (Blanke and Lorentzen,2006;Frisk et al.,2017). The computational forms for each PC can be implemented in several ways: for instance, using the proposed steps directly as a simulation model, as in the DX approach (Pulido and Alonso-González,2004), as a state-observer (Bregon et al.,2014), or as a particle-filter using a probabilistic model (Alonso-Gonzalez et al.,2011). However, they can also be used to build estimation models for fault identification (Bregon et al.,2012), or just to provide the structural model for building data-driven gray-box models (Pulido et al.,2019). In the online stage, the computational model for each PC is fed with observations to produce an estimation, and consequently to compute a residual, which can be used for fault detection. If the residual is significantly different from zero, the Possible Conflict becomes a minimal conflict in the DX terminology, and the structural information – i.e., the correctness assumptions in DX terminology – can be used later to perform fault isolation. This stage is straightforward following Reiter’s theory (Reiter,1987): diagnosis candidates are obtained by computing the minimal hitting sets of the sets of equations in each PC. We illustrate this process in our case study next. 4. Generating Possible Conflicts for a case study 4.1. Case study In order to illustrate the proposal of this paper, and later evaluate its performance in a real world situation, we use a laboratory plant that consists of two cylindrical tanks, T1and T2, both with the same crosssectional area. These tanks are connected by a cylindrical pipe with a manual valve. Fig. 1 shows the scheme of this plant. 4The tool for computing the set of PCs from an abstract representation of the system model and the software to perform fault diagnosis using PCs can be found at https://www.infor.uva.es/softwarepcs/ 4 D. Garcia-Alvarez, A. Bregon, B. Pulido et al. Engineering Applications of Artificial Intelligence 122 (2023) 106145 Tank levels are measured using two level sensors LT1and LT2. The liquid is drained from the tanks T1and T2by means of two pipes, p10 and p20, with flows 𝑞10 and 𝑞20, respectively. The two tanks are connected by pipe p12, whose flow is 𝑞12. Both tank levels are controlled, but the control system is not considered in neither the plant model nor the fault detection technique. A detailed description of the hardware and software of this plant can be found in Fuente et al. (2008). In this system, we have considered typical faults, such as sensor faults, tank leakages, and clogged pipes. We deal with this issue later in this section. This plant can be modeled using only Physics First Principles equations. The reader should notice that we provide only models of correct behavior for each equation. These models do not include additional (internal and non-observable) variables, which can be useful to model faulty behaviors, such as flows related to leakages in tanks, additional parameters to model potential pipe clogs, or potential bias in sensors. Additionally, our models obey the locality and no-function-in structure principles in Consistency-based Diagnosis (Reiter,1987;Hamscher et al.,1992), and each equation is sensitive to exactly one fault.5 Consequently, our behavior models are as follows. First, the change in the level in each tank can be formulated according to the mass balances: 𝐴 ℎ1(𝑡) = 𝑞∗ 1(𝑡) − 𝑞12(𝑡) − 𝑞10(𝑡)(eq1) 𝐴 ℎ2(𝑡) = 𝑞∗ 2(𝑡) + 𝑞12(𝑡) − 𝑞20(𝑡)(eq2) where 𝐴is the area of the cylindrical tanks (𝐴= 314 cm2), ℎ1(𝑡)and ℎ2(𝑡)are the liquid level in each tank (  ℎ(𝑡)is the derivative), and 𝑞∗ 1(𝑡) and 𝑞∗ 2(𝑡)the input flow of each tank. Variables marked with an asterisk (*) are known variables, such as measured variables or known inputs. The maximum level of the tanks (ℎ𝑚𝑎𝑥) is 36.3 cm. Following Torricelli’s law, flows 𝑞12,𝑞10 and 𝑞20 are calculated by: 𝑞12(𝑡) = 𝐾12 sign(ℎ1(𝑡) − ℎ2(𝑡))√2𝑔||ℎ1(𝑡) − ℎ2(𝑡)||(eq3) 𝑞10(𝑡) = 𝐾10√2𝑔ℎ1(𝑡)(eq4) 𝑞20(𝑡) = 𝐾20√2𝑔ℎ2(𝑡)(eq5) where 𝐾12 = 0.0499 cm2,𝐾10 = 0.097 cm2and 𝐾20 = 0.09665 cm2are the cross sectional area of the pipes and 𝑔is the gravitational acceleration 𝑔= 9.8m∕s2). The observational model, which relates internal and measured variables, is given by the following equations: ℎ∗ 1(𝑡) = ℎ1(𝑡)(eq6) ℎ∗ 2(𝑡) = ℎ2(𝑡)(eq7) where ℎ∗ 1(𝑡)and ℎ∗ 2(𝑡)are the sensor measurements corresponding to LT1 and LT2. The relationship between the state variables (  ℎ1and  ℎ2) and their derivatives is established through equations: ℎ1(𝑡+ 1) = ∫ ℎ1(𝑡)𝑑𝑡 (eq8) ℎ2(𝑡+ 1) = ∫ ℎ2(𝑡)𝑑𝑡 (eq9) Flows 𝑞∗ 1and 𝑞∗ 2, provided by pumps P1and P2, are considered as known inputs in this example. The model was fitted by estimating 5This assumption could be removed in other systems. If the models have equations that can be related to more than one fault, we can still use them. Each PC containing those types of equations should be sensitive to those faults. This is neither relevant for PC calculation nor fault isolation. It would only be relevant for fault identification where we need to discriminate between those faults. Table 1 List of equations and related faults. Equation Variables Component Fault Description eq1{ ℎ1,𝑞∗ 1,𝑞12,𝑞10} T1fT1Leakage in tank T1 eq2{ ℎ2,𝑞∗ 2,𝑞12,𝑞20} T2fT2Leakage in tank T2 eq3{𝑞12,ℎ1,ℎ2} p12 fp12 Blockage of pipe/valve p12 eq4{𝑞10,ℎ1} p10 fp10 Blockage of pipe/valve p10 eq5{𝑞20,ℎ2} p20 fp20 Blockage of pipe/valve p20 eq6{ ℎ1,ℎ∗ 1} LT1fLT1Faulty sensor LT1 eq7{ ℎ2,ℎ∗ 2} LT2fLT2Faulty sensor LT2 eq8{ ℎ1,ℎ1} – – – eq9{ ℎ2,ℎ2} – – – parameters 𝐾12,𝐾10 and 𝐾20 using real data collected from the plant. The process to fit these parameters was presented in García-Álvarez et al. (2011). In order to test the approach presented in this work, a specific plant configuration has been set. It consists of the following behavior: •Tank 1 (T1): the level reference is fixed at 30%. •Tank 2 (T2): the level reference is fixed at 30% and then changed to 50% at time sample 800 s. Both tanks are empty at the beginning of each experiment. The sampling time is 1s and all performed experiments have a duration of 1600 s. This experiment allows the plant to run through 4 different states, two transient states and two steady states: 1. Transient State 1: both tanks are filled to reach the reference level. 2. Steady State 1: both tanks maintain the level at 30%. 3. Transient State 2: tank T2is filled to reach the reference 50%. 4. Steady State 2: T1maintains the level at 30% and T2maintains the level at 50%. Fig. 2 shows the evolution of a nominal experiment considering the four states. Tank levels are expressed in percentages with respect to ℎ𝑚𝑎𝑥. 4.2. PCs for the case study Regarding our case study, the relevant information for PCs computation can be found in Table 1: we only need to known the set of known and unknown variables for each equation, together with the physical component or the fault behavior which is being modeled. Additionally, Table 1 provides a brief description of the set of faults considered for the case study: leakages in both tanks, clogged pipes, and bias in both level sensors. As can be seen, each equation from eq1to eq7is linked to exactly one fault in the system. For instance, equations eq1and eq2could be used to model leakages in tanks 𝑇1and 𝑇2by merely subtracting the non-observable variables 𝑓𝑙𝑒𝑎𝑘1and 𝑓𝑙𝑒𝑎𝑘2 to simulate the leakage flows 𝑓𝑇1and 𝑓𝑇2, respectively. In a similar way, the parameters 𝐾12, 𝐾10, 𝐾20, would relate equations eq3, eq4, and eq5 exactly to one fault: 𝑓𝑝12 , 𝑓𝑝10 , and 𝑓𝑝20 , respectively (just multiplying by the unknown faulty parameters 𝑘𝑓12 , 𝑘𝑓10 , 𝑘𝑓20 : if 𝑘𝑓𝑖is 1, there is no fault. If 𝑘𝑓𝑖moves closer to 0, we have a smaller or larger blockage). Finally, equations eq6and eq7can be used to model faults in the level sensors (bias) 𝑓𝐿𝑇1and 𝑓𝐿𝑇2in a similar way to leakages in equations eq1or eq2. There is no faulty behavior modeled by equations eq8, and eq9, which only model the dynamic behavior of the system. Using just the information in columns 1 and 2 in Table 1, the software used to compute the set of PCs follows the steps described in Section 3. We obtain 4 PCs that can be seen in Table 2, together with the set of faults related to these equations, the estimated variable for each PC, and the set of equations. Fig. 3 provides the set of components, and the computational form for PC1, which is represented by the directed hypergraph in the right 5 D. Garcia-Alvarez, A. Bregon, B. Pulido et al. Engineering Applications of Artificial Intelligence 122 (2023) 106145 Fig. 2. Evolution of measured variables (ℎ∗ 1and ℎ∗ 2) and inputs (𝑞∗ 1and 𝑞∗ 2). Fig. 3. Variables, measurements and components related to the directed hypergraph for PC1. Table 2 PCs and their related equations, estimated variables, and fault modes. PC Equations in the PC Estimated variable Faults in the PC fT1fT2fp12 fp10 fp20 fLT1fLT2 PC1{eq1,eq3,eq4,eq6,eq7,eq8}ℎ11 0 1 1 0 1 1 PC2{eq1,eq2,eq3,eq4,eq6,eq8}ℎ11 1 1 1 0 1 0 PC3{eq2,eq3,eq5,eq6,eq7,eq9}ℎ20 1 1 0 1 1 1 PC4{eq1,eq2,eq3,eq5,eq7,eq8,eq9}ℎ21 1 1 0 1 0 1 hand side of the figure. Following the hypergraph, the equations are solved from the leaves to the root: ℎ1, which can be either measured, using LT1, or estimated using the model. To estimate ℎ1, we need the value of its derivative,  ℎ1, which can be computed if 𝑞∗ 1,𝑞10, and 𝑞12 are known. 𝑞∗ 1is an input. 𝑞10 can be computed if ℎ1is known. 𝑞12 needs ℎ2(which is given by LT2). The variable in the root is the only estimate for the PC, as expressed in Table 2. The difference between the real measurement and the estimation for that variable generates a residual signal for each PC. This residual could be used to perform fault detection. As an example of the usage of PCs for FDII, let us assume that there is a fault in sensor LT2, i.e., 𝑓𝐿𝑇2is present. Sooner or later, every PC related to 𝑓𝐿𝑇2will be activated. Let us assume that the first PC to be activated is PC4. The CBD approach using PCs proceeds by analyzing Table 2 row-wise: faults related to the equations in the PC ({eq1,eq2,eq3,eq5,eq7,eq8,eq9}) can be responsible for the fault. 6 D. Garcia-Alvarez, A. Bregon, B. Pulido et al. Engineering Applications of Artificial Intelligence 122 (2023) 106145 Following CBD minimal conflicts are computed as minimal hitting sets from the sets of PCs. Hence, since there is no previous fault candidates the minimal hitting set for these equations provides the set of single fault candidates: [[𝑓𝑇1],[𝑓𝑇2],[𝑓𝑝12 ],[𝑓𝑝20 ],[𝑓𝐿𝑇2]]. If PC1is activated later, it would signal to the following equations as candidates for the fault: {eq1, eq3, eq4, eq6, eq7, eq8}. These equations are related to single fault candidates [[𝑓𝑇1],[𝑓𝑝12 ],[𝑓𝑝10 ],[𝑓𝐿𝑇1],[𝑓𝐿2]] However, this that must be consistent with the previous activation of PC4. Hence, the set of consistent faults will be related to the minimalhitting sets of the equations of both PCs: [[eq1], [eq3], [eq7], [eq2,eq4], [eq2,eq6], [eq4,eq5], [eq5,eq6]]. These equations correspond to 3 single faults: [[𝑓𝑇1],[𝑓𝑃12 ],[𝑓𝐿𝑇2], and 4 double faults: [𝑓𝑇2, 𝑓𝐿𝑇1],[𝑓𝑝10 , 𝑓𝑝20 ], [𝑓𝑝20 , 𝑓𝐿𝑇1],[𝑓𝑇2, 𝑓𝑝10 ]] Finally, when 𝑃 𝐶3is activated, we follow the same procedure, computing new minimal-hitting sets for the three PCs. The set of equations in the minimal-hitting sets are: [[eq3], [eq7], [eq1,eq2], [eq1,eq5], [eq1,eq6], [eq2,eq4], [eq2,eq6], [eq4,eq5], [eq5,eq6]]. The corresponding set of fault modes are: [[𝑓𝑝12 ],[𝑓𝐿𝑇2],[𝑓𝑇1, 𝑓𝑇2],[𝑓𝑇1, 𝑓𝑝20 ], [𝑓𝑇1, 𝑓𝐿𝑇1],[𝑓𝑇2, 𝑓𝑝10 ],[𝑓𝑇2, 𝑓𝐿𝑇1],[𝑓𝑝10 , 𝑓𝑝20 ],[𝑓𝑝20 , 𝑓𝐿𝑇1]]. As the reader can see, even without information about the effect of the fault modes, the fault in sensor LT2is among the potential single fault candidates. Usually, a fault identification stage is included in the CBD approach to help refine further the set of fault modes consistent with the observations. That is, however, beyond the scope of this work. It must also be mentioned that the final result of single and double (or multiple) faults is the same, whatever the order of activation for the PCs. The procedure is automatic, and there is no assumption about the number of double or multiple faults. All of them are computed. 5. Integration approach and discussion PCA-based fault detection techniques offer many advantages. These methods are easy and straightforward to implement and only require a good database containing the main trends in the system, regardless of whether the data are highly correlated. In addition, the detection task has a low computational cost, and can be completed by identifying which measured variables are directly or indirectly related to the fault thanks to contribution analysis. However, as mentioned in previous sections these methods have some weaknesses too. Regarding fault isolation, classical PCA approaches do not provide tools to isolate faulty candidates when a fault is detected in the process. There are several authors who have proposed solutions for this flaw. Gertler et al. (1999) uses structured partial PCA models with the same isolation properties as the parity relations. Using that equivalence, Gertler et al. (1999) and Huang et al. (2000) decompose the original PCA model into smaller structured PCA models that guarantee the disturbance decoupling for the set of faults considered. Another flaw that can be remarked is the use of classical PCA approaches in processes with several operation points. In general, these methods are used in processes without many operation points (GarcíaÁlvarez et al.,2012). When dealing with processes with several operation points, Hwang and Han (1999) and Tien et al. (2004) classify the different solutions into three different categories: 1. Building a PCA model for each operation point: This family of methods is known as Multiple PCA (MPCA) or MPLS (Multiple Projection to Latent Structures) in Zhao et al. (2004,2006), where the authors propose using principal angles to distinguish the operating modes. Solutions falling into this category can be compromised if the process has many operation points, because they will need to fit a PCA model for each one of them. In addition, the complex task of monitoring transient states must be considered. A method to deal with transient states using PCA can be found in García-Álvarez et al. (2012) which is similar to the procedure used by PCA-based techniques for monitoring batch process (Nomikos and MacGregor,1995;Zarzo and Ferrer, 2004). These methods will require additional PCA models for each of the transient states. 2. Updating the model to reflect the changes in the operation points: this category aims to adapt the PCA model to the current operation point. This family of methods can include such configurations as Exponentially Weighted PCA (EWPCA) (Lane et al.,2003), (Portnoy et al.,2016), Adaptive PCA (APCA) (Bakdi and Kouadri,2017), Moving Window PCA (Rafferty et al.,2016), Dynamic PCA (DPCA) (De Ketelaere et al.,2015), and Recursive PCA (RPCA) (Rezamand et al.,2020). These proposals require the on-line reformulation of the PCA model with every change detection in the operation point. The fault detection system must detect the change in the current operating point to avoid its interpretation as a fault and reformulate the PCA model to capture the trends of the new point. 3. Developing a conventional PCA model to capture all such changes. One of the main drawbacks of methods of this category is that PCA could identify transient states between operation points as faults due to their non-linear behavior. Furthermore, if the number of operation points is high, the PCA model can capture an excessive number of tendencies. If the PCA model is fitted considering the tendencies of each case, the PCA monitoring technique can model too many tendencies in the nominal case, and the PCA-based fault detection method could exhibit poor detectability due to the difficulty to distinguish between faults and operation points. Finally, there are hybrid solutions that combine data-driven and model-based approaches. These methods are capable of generating and/or processing residual signals obtained by using model-based fault detection. For instance, the work presented in Jiang et al. (2021), which proposes constructing optimized residual generators; Jung and Frisk (2018), where a residual selection approach is proposed using structural information and training data from different fault scenarios; and Jung and Sundström (2019), where a comparative study of model-based, data-driven and hybrid techniques in fault detection and diagnosis tasks is presented. The integration approach presented in this work fits within this last group of techniques, because we propose to use a PCA model to track not the directed measurements from the system, but the residuals generated by PCs. This approach is based on two basic ideas: on the one hand, PCA-based fault detection approaches are straightforward to implement and the on-line detection presents a low computational cost. Using PCA, the process can be monitored by simply looking at the 𝑇2and 𝑄statistics. On the other hand, the PCs technique goes one step further in the fault detection and isolation tasks by looking at the triggered residual signals. 5.1. How the integration approach works Fig. 4 presents a graphical scheme of the integration approach presented in this work. This scheme can be divided into two main procedures. First, the off-line procedure, where the different models required for the detection and diagnosis tasks are established. This procedure is represented at the bottom of this figure in blue boxes. Second the on-line procedure, where the fault detection and isolation tasks are run. This process is represented at the top of this figure in orange boxes. The off-line procedure consists of the following tasks: 1. Obtain two abstract descriptions from the process model (the non-directed hypergraph to build the set of MECs, and the directed hypergraph to build the set of MEMs for the set of MECs). 2. Compute the set of PCs (those MECs with at least one MEM) and build their computational models. 3. Analyze the fault information in each PC to build the theoretical Fault Signature Matrix (FSM). 7 D. Garcia-Alvarez, A. Bregon, B. Pulido et al. Engineering Applications of Artificial Intelligence 122 (2023) 106145 Fig. 4. Integration proposal. At the bottom of the figure, the off-line computation modules are represented in blue. At the top, the different modules which are run during the on-line process monitoring are shown in orange. The fault detection and fault isolation tasks are marked in the scheme in yellow boxes. Blocks Normalized error contribution, Normalized score contribution and Overall average variable contribution are computed through the contribution analysis explained in Section 2. 4. Fit the PCA model using the historical values of residual signals for the PCs using NOC data. The on-line procedure consists of the following tasks: 1. Initialize the set of active fault candidates to empty. 2. While there are new observations available and there is at least one PC not activated do {Fault detection:} (a) For each PC𝑖do Compute its residual signal 𝑟𝑖using available observations. (b) Monitor the calculated residual signals by means of the 𝑇2and 𝑄statistics. (c) If either the 𝑇2or 𝑄statistics exceed their thresholds for a certain number of consecutive samples then confirm the activation of PC𝑖. •If the 𝑄statistic is responsible for the detection, then compute the residual contribution using the Bar plot of normalized errors of the variables (See Section 2) and mark PCs with high related residuals as activated PCs. •If the 𝑇2statistic is responsible for the detection then compute the residual contribution using the Bar plot of normalized scores and the Overall average variable contribution of the highest scores (See Section 2) and mark PCs with high related residuals as activated PCs. {Fault isolation:} (d) If there is a new activated PC then: update the set of active fault candidates (analyzing the theoretical FSM row-wise and computing the minimal hitting-set). The reader should notice that the activation of either the 𝑄or the 𝑇2 statistic will activate the corresponding PCs, because we are following the consistency-based diagnosis approach to find the fault candidates, and these fault candidates must be consistent with the whole set of activated conflicts. 5.2. Discussion If we compare this proposal with those mentioned at the beginning of this section we can point out several relevant differences and some novelties: •Considering the PCA-based methods mentioned above, this proposal does not need to fit different PCA models for different operation points as the MPCA-based methods (Zhao et al.,2004, 2006) propose. Related to PCA approaches that are able to adapt to the current operation point (Lane et al.,2003;Portnoy et al., 2016;Bakdi and Kouadri,2017;Rafferty et al.,2016;De Ketelaere et al.,2015;Rezamand et al.,2020), the main difference is that we only require correct NOC data and we only need to build one PCA model, offline, instead of recomputing the model or adapting the monitoring task online when there is a change in the operation point. •Regarding techniques which adapt the PCA model to new operation points, such as EWPCA (Lane et al.,2003;Portnoy et al., 2016), DPCA (De Ketelaere et al.,2015), Moving Window PCA (Rafferty et al.,2016) or RPCA (Rezamand et al.,2020), the main difference is again that we need only to fit one PCA model offline, using the NOC data concerning normal behavior. Since the PCs will track the changes in the operation point and the PCA model does not need to be aware of that, we do not need to compute the model again every time there is a change in the operation point. •Regarding the set of approaches that combine data-driven models and model-based diagnosis (using mainly first principles or other analytical models): we use only one PCA model that analyzes residual signals for the whole system, instead of using Partial PCAs, created for structured residuals that are only sensitive to some faults, to direct measurements from the system or other derived signals (Gertler et al.,1999). Additionally, our proposal relies exclusively upon correct behavior models; thus, we only need correct behavior data and we do not need faulty behavior data (Jung and Frisk,2018;Jung and Sundström,2019). At the same time, using PCs we can track different operation points without considering different PCA models for each one of them. Regarding the work by Jiang et al. (2021), our proposal does not propose an optimization method to select the most sensitive residuals provided by parity relations (similar in many ways to ARRs). We use the residuals based on minimality with respect to the available model and use another statistical model, PCA. Last, but not least, PCs are used for fault isolation using the CBD 8 D. Garcia-Alvarez, A. Bregon, B. Pulido et al. Engineering Applications of Artificial Intelligence 122 (2023) 106145 principles, which are different and more sound with respect to the fault isolation methods used by the FDI community (Cordier et al.,2004). •Finally, we have tested the behavior of traditional Artificial Neural Networks (ANN) solutions (Multilayer perceptron) training two Neural Networks (NN) in cascade configuration, one for fault detection (that must avoid false positives due to operation point changes) and another one for fault isolation (capable to distinguish between two sensor faults). Samples are introduced into the first NN for fault detection. If a detection is established, the sample is processed by the second NN, which provides fault isolation information. Both NN solve a classification problem. We cope with the system dynamics using a semi-dynamical architecture, which requires feeding the network with current and past data for each variable, using a sliding window approach. We selected 80% of the data for training the models. The remaining 20% for testing the models. We made a stratified random partition, to ensure that both the training and test sets had at least one sample of each type of failure. We used a 5-folds stratified cross validation process on the training set to select the best ANN model. We tried different architectures (number and size of layers) epochs and batch sizes and build the final model with the best parameters with the full training set. The final model has 60 elements in the input layer, a hidden layer with 40 units and an output layer with one element. Experimental results shown that in the nominal experiments of the test dataset, the detection NN generate 8 false positives, which is not competitive with the PCA-PCs approach. The NN is sensitive to the first operation mode change, while the PCA+PCs is not. 6. Results In this section, we illustrate how the proposed methodology can be used in our case study. First, we show how, by using PCs, we can obtain residuals from NOC historical data and compute a PCA model which drastically reduces the number of false positives in fault detection. We then show how this PCA model, built completely offline, can be used online to detect faults and successfully provide fault isolation results using structural information from the PCs with activated residuals. 6.1. Nominal case In order to study the results obtained using the method proposed in this work, two PCA models were fitted. The first one consisted of a classical PCA approach. In this case, the PCA model was fitted using the measured variables (ℎ∗ 1and ℎ∗ 2) and the inputs (𝑞∗ 1and 𝑞∗ 2) collected from 10 real experiments in NOC. Each experiment included the four described states (transient and steady). The PCA model was fitted with two principal components. The detection thresholds for the PCA model were established with a level of significance 𝛼= 95%. This configuration is denominated the PCA method in this work. The second one was fitted following the integration approach presented in this paper. The PCA model of the integration approach, from now on the PCA+PCs method, was fitted using the data of the residual signals (𝑟1,𝑟2,𝑟3and 𝑟4) of each PC calculated for the same 10 real experiments used in the PCA method. This PCA model was also fitted with two principal components and the detection thresholds were also established with the same level of significance 𝛼= 95%. The PC model simulation and the PCA computation were developed with Matlab© and Simulink©. It is interesting to notice the behavior of the residual signals (𝑟1, 𝑟2,𝑟3, and 𝑟4) related to each identified PC in NOC in Fig. 5. These residual signals were computed using the data of the experiment shown in Fig. 2. As Fig. 5 shows, the evolution of the different residuals can be affected by the current operation point or transient state. In this case, the deviation of the residual signals could be due to the Table 3 Mean value and standard deviation of the alarms percentage obtained for 20 experiments under nominal conditions. # Alarm percentage PCA PC+PCA 1 7.37 2.06 2 7.80 3.25 3 7.06 1.69 4 6.99 0.68 5 7.93 3.44 6 7.06 3.43 7 7.25 1.12 8 7.81 1.37 9 7.93 1.99 10 7.62 1.87 11 8.62 3.74 12 8.06 0.74 13 7.62 1.18 14 7.99 1.18 15 7.75 1.99 16 7.43 3.31 17 8.81 1.18 18 7.62 1.37 19 7.81 2.18 20 8.50 2.43 Mean 7.75 2.01 Std 0.50 0.96 ANOVA data F-ratio 558.71 P-value 0.000001 fact that the estimated model parameters are more accurate in linear operation points than in transitory states, whose inner relationships between variables are typically non linear. The first contribution of this methodology can be seen by studying the behavior of the system in the nominal situation. Considering the monitoring and fault detection tasks, an alarm can be defined as a time sample where the value of 𝑇2or 𝑄gives a value greater than its related threshold. In this context, one alarm is not enough to detect a fault; several alarms above the fixed threshold or a given ratio of alarms in a given period of time are two methods to determine a fault detection. In this work, the number of consecutive alarms that must occur for a fault to be detected has been empirically set to 15. When a fault is detected in a non-faulty situation, it is said that a false positive (FP) has been detected. In the MSPC context, it is normal to have alarms in the different control charts. However reducing the number of alarms can be a positive measure from the point of view of the cognitive overload of plant operators. Table 3 shows the percentage of alarms detected with PCA and PCA+PCs methods. Looking at Table 3, the reduction in the number of alarms can be clearly seen when the PCA+PCs is used. After performing an analysis of variance (ANOVA) for comparing both mean values, the P-value of the F-ratio is lower than 0.05, so there exists a statistically significant difference between the means of the methods at the 95.0% confidence level. Regarding the false positive detection, when the PCA method was applied, a false positive was detected in transitory state 1in each of the 20 experiments studied. With the PCs alone (just paired with a ztest to identify significant deviations of the residual signals), at least one false positive was triggered in each experiment. However, when the PCA+PCs method is applied, no false positive was detected in any of the experiments. Table 4 shows a summary of the number of false positives detected by the PCA, PCs and PCA + PCs methods using the same NOC experiments as in Table 3. Fig. 6 shows the evolution of the control charts based on the 𝑇2 and 𝑄statistics for the first experiment of Table 3. In this figure, it can be clearly seen that the start-up (when both tanks are being filled) is detected as a fault by both statistics when the PCA method is applied (Fig. 6(a)). However, when the PCA+PCs method is used, no fault is detected (Fig. 6(b)). 9