Explainable unsupervised statistical learning for online anomaly detection in oil wells

Explainable unsupervised statistical learning for online anomaly detection in oil wells Rodrigo Marcel Araujo Oliveira 1 Ângelo Márcio Oliveira Sant’Anna 2 Paulo Henrique Ferreira da Silva 3 Abstract The occurrence of failures in oil production wells can cause financial losses and catastrophic environmental damage. This article proposes a prognostic tool for anomaly detection using unsupervised learning models. The SHAP method was applied to the iForest model, which has allowed for achieving satisfactory results for online anomaly detection, indicating the operational risk to each monitoring process variable. keywords: statistical machine learning; unsupervised learning; anomaly detection; explainable artificial intelligence; oil wells. 1 Introduction In the oil and gas industry, driven by economic, environmental, and regulatory factors, monitoring processes to ensure operational safety is essential. Oil and gas extraction involves complex structures and requires efficient management to analyze a large set of information from various parts of the system that make up an oil well. A set of electrical, mechanical sensors, and hydraulic systems structures constitute an oil well (Vargas et al., 2019). These processes are complex and are subject to failures, which can vary according to the structure of the oil well. The occurrence of failures in oil production wells can cause financial losses and catastrophic environmental damage. Process monitoring with real-time data allows the development of methodologies for anomaly prognosis in drilling and oil extraction operations, which can yield benefits contributing to decision-making processes. Statistical Machine Learning is a branch of Artificial Intelligence (AI) consisting of powerful algorithms capable of recognizing complex patterns based on data (Hastie et al., 2009). These algorithms have been gaining traction in academia for detecting anomalous events. Anomaly detection is the recognition of patterns that do not conform to expected behavior. The approach using the unsupervised Local Outlier Factor (LOF) model for anomaly detection in oil wells was proposed by Aranha et al. (2023). The authors developed several models for different time windows and oil wells. The results were comparable to other studies using the same dataset, such as the Long Short-Term Memory (LSTM) and Support Vector Machine (SVM) models proposed by Machado et al. (2022), enabling better performance for dynamic fault detection times. Marins et al. (2021) investigated Bayesian models and Random Forest for conditionbased monitoring construction. These methods enabled minimal delay detection, providing sufficient time for data mitigation. Turan and Jaschke (2021) presented different model 1Polytechnic School, PEI-UFBA - ro[email protected] 2Polytechnic School, PEI-UFBA - [email protected] 3Institute of Mathematics and Statistics, PGECD-UFBA - [email protected] approaches for fault classification in oil wells with emphasis on Decision Tree, Random Forest, and AdaBoost models, which achieved the best performances. Vargas et al. (2017) proposed utilizing techniques such as K-Nearest Neighbors (KNN), t-distributed Stochastic Neighbor Embedding (t-SNE), and multi-scale sliding window algorithms to detect production issues in oil wells such as: lower hole safety valve; loss of natural flow; production line obstruction; choke valve closure. The feasibility of these methodologies using real and simulated data showed robust results in applications with actual production plants. The oil and gas industry requires approaches capable of detecting faults in a short period for damage mitigation. This work evaluates different methods for online anomaly detection to multivariate time series data. The research presents an approach based on Explainable Artificial Intelligence (XAI) to identify the variables with the greatest influences for discriminating anomalies, with the assistance of the Isolation Forest (iForest) model and the SHapley Additive exPlanations (SHAP) technique. This enables effective operation in identifying process faults, with results showing potential to improve efficiency and operational safety in the oil industry. 2 Experimental Data Brazil’s largest oil and gas company has made available a dataset containing information from sensor variables used in monitoring processes of emerging offshore oil wells. The 3W dataset comprises multiple pressure and temperature sensors (Vargas et al., 2019). The data motivation consists of monitoring information to detect anomalies in offshore oil wells operated by natural lift. The sensors include: fluid pressure from the Permanent Downhole Gauge (P-PDG); fluid pressure and temperature at the Temperature and Pressure Transducer (P-TPT and T-TPT); fluid pressure at the Production Choke Valve (P-MON-CKP); temperature downstream of the Production Choke Valve (T-JUS-CKP); variable pressure upstream of the gas lift choke (P-JUS-CKGL); variable temperature upstream of the gas lift choke (T-JUS-CKGL); gas lift flow rate (QGL). The units of measurement for pressure and temperature are, respectively, Pa and ◦C. Anomalies are classified by experts about oil lifting and flow, as described by Vargas et al. (2019). The dataset comprises three categories of data: real, simulated, and synthetic. This work is limited to real data only. The historical data consists of three types of periods: normal, transient, and anomaly. For model development, the transient data was considered as anomalous, aiming to develop models that serve for anomaly detection prognosis rather than just diagnosis. In this scenario, intervention in the failure process is possible. The real data sample contains 14,516,197 observations, with 9,439,612 observations exhibiting normal patterns and 5,076,585 anomalous observations. 3 Methodology 3.1 Isolation Forest The iForest proposed by Liu et al. (2008) is an unsupervised model for anomaly detection based on a set of binary decision trees. The model involves making random data partitions to form isolation trees. For each novel point, the algorithm generates a score, represented by Equation (1), where E(h(x)) is the average depth that a novel point reaches across all trees in the forest. The normalization factor c(k)represents the average unsuccessful depth in a binary tree search (2), with kdenoting the number of points used in tree construction, and the function H(i)is the estimated harmonic number (3). S(x, k)=2−E(h(x)) c(x),(1) c(k)=2H(k−1) −2(k−1) k,(2) H(i) = ln(i) + 0,5772156649.(3) 3.2 Principal Component Analysis Principal Component Analysis (PCA) consists of linear dimensionality reduction using singular value decomposition of the data to project them into a lower-dimensional space. Shyu et al. (2003) proposed an approach for anomaly detection using PCA. The method involves calculating a covariance matrix of the data, which is decomposed by orthogonal vectors, called eigenvectors, associated with eigenvalues. Most of the variation in the data is captured by eigenvectors with high eigenvalues. Therefore, outliers are characterized in the hyperplane by eigenvectors with small eigenvalues. The algorithm generates a score based on the sum of the projected distance of a sample on all eigenvectors. 3.3 Local Outlier Factor The LOF algorithm proposed by Breunig et al. (2000) is an anomaly detection method that involves calculating the local density deviation of a particular point relative to its neighboring points. The estimation of local density is obtained by calculating distances between the k-nearest neighbors (4). The method compares which points have a lower density than their neighbors, and these distances are used to calculate the reachability distance, defined as the maximum of the distance between two points and the k-distance relative to the reference point (5). The reachability distance determines how close the points are. The LOF score is obtained by the ratio between the average of local reachability densities, the number kof neighbors, and the local reachability density of the point (6). Rk(i, j) = max {dk(i), dk(i, j)},(4) LRDk(i) = 1 Pj∈Nk(i)Rk(i,j) |Nk(i)|,(5) LOFk(i) = Pj∈Nk(i)Rk(i, j)LRDk(j) LRDk(i) |Nk(i)|.(6) 3.4 Lightweight on-line detector of anomalies The Lightweight On-line Detector of Anomalies (LODA) algorithm proposed by Pevný (2016) consists of a collection of kone-dimensional histograms {hi}k i=1, each approximating the probability density of the input data projected onto a single projection vector {wi∈ Rd}k i=1. The LODA score is an average of the logarithm of estimated probabilities in individual projection vectors, described by Equation (7), where ˆpidenotes the probability estimated by the i-th histogram and widenotes the corresponding projection vector. f(x) = −log   k Y j=1 ˆpi(x⊤wi)  1 k .(7) 3.5 SHapley Additive exPlanations The SHAP method proposed by Lundberg et al. (2017) is notably one of the most widely used techniques for explaining machine learning models. This approach is based on Game Theory and incorporates methods for global and local interpretations such as feature importance, feature dependence, and interactions. 4 Results and Discussion This project was developed with the aid of the Python program. The data was divided into two sets of samples, taking into account the temporal order of the data for each oil well, with 80% for training and 20% for testing, representing, respectively, 8,887,177 and 2,221,800 samples, null values were discarded. Data standardization using the Gaussian distribution N(0,1) was adopted due to the differences in the magnitudes of the predictor variables. The performance evaluation of the fitted models was based on metrics derived from the confusion matrix: Accuracy; Precision; Recall; Specificity; F1-score; AUC-ROC (Hastie et al., 2009). The contamination factor of the anomaly detection models consists of the percentage of anomalous events in the training sample, which corresponds to 30%. The configuration adopted for the iForest model in modeling was: 256 for the number of trees in the forest; 100% for the maximum number of variables in each tree. For PCA, all principal components were considered in the model composition. For LOF, the considered distance was Euclidean with 7 nearest neighbors. The LODA parameters were: number of histogram bins and the number of random cuts, both set to 1,000. Table 1 presents the results of the models for the training and test sets. The iForest and PCA models for both evaluation sets achieved the best performances. The iForest model achieved the best Recall metrics, which is interesting as it classifies most anomalous observations before oil production failure occurs. For the interpretability analysis of the results, the iForest model was considered, as the SHAP method is ideal for tree-based models. Figure 1 provides a global interpretation of the results, where each point on the graph represents an observation from the sample. It is possible to identify the impact of each variable on the model’s decision in anomaly detection according to the color shade. For the variable P-PDG, high values contribute to a higher chance of the observation being an anomaly, and the same applies to low values of the variable T-TPT. A random point from the test set with a normal pattern was selected for a Local Interpretation. In Figure 2, the gray axis corresponds to the impact of each variable on the model result, where it is noticeable that for variable P-PDG equal to 0.145 it contributed to a greater chance of the observation being an anomaly. Table 1: Evaluation metrics. Models Dataset Accuracy Precision Recall Specificity F1-score AUC-ROC iForest train 80.49% 71.84% 69.89% 85.94% 70.85% 77.91% test 73.45% 55.25% 100.00% 60.51% 71.17% 80.26% PCA train 75.30% 63.98% 62.23% 80.01% 63.09% 72.12% test 87.75% 78.13% 86.96% 88.13% 82.31% 87.75% LODA train 74.07% 62.12% 60.42% 81.08% 61.26% 70.75% test 64.79% 47.65% 75.10% 59.77% 58.30% 67.44% LOF train 59.61% 39.35% 35.22% 72.14% 37.17% 53.68% test 53.50% 40.86% 94.98% 33.41% 57.14% 64.20% Figure 1: Global explainability with SHAP for iForest model. Figure 2: Local explainability with SHAP for iForest model. 5 Conclusions In this article, a prognostic tool for anomaly detection was proposed using unsupervised learning models. The iForest models presented robust results, particularly for handling many samples that need to be processed in real time. The SHAP method made it possible to identify which variables are most important for discriminating anomalous events, allowing for real-time prognosis in oil wells, which is essential for mitigating risks and anticipating potential production failures that could be catastrophic financially, environmentally, and for human life. Acknowledgments This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) – Finance Code 001. References Aranha, P. E., N. A. Policarpo, and M. A. Sampaio (2023, 11). Unsupervised machine learning model for predicting anomalies in subsurface safety valves and application in offshore wells during oil production. Journal of Petroleum Exploration and Production Technology 14(2), 567–581. Breunig, M. M., H. P. Kriegel, R. T. Ng, and J. Sander (2000). LOF: Identifying DensityBased Local Outliers. SIGMOD 2000 - Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, 93–104. Hastie, T., R. Tibshirani, and J. Friedman (2009). The Elements of Statistical Learning. Liu, F. T., K. M. Ting, and Z. H. Zhou (2008). Isolation forest. Proceedings - IEEE International Conference on Data Mining, ICDM , 413–422. Lundberg, S. M., P. G. Allen, and S.-I. Lee (2017). A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems 30. Machado, A. P. F., R. E. V. Vargas, P. M. Ciarelli, and C. J. Munaro (2022, 11). Improving performance of one-class classifiers applied to anomaly detection in oil wells. Journal of Petroleum Science and Engineering 218, 110983. Marins, M. A., B. D. Barros, I. H. Santos, D. C. Barrionuevo, R. E. Vargas, T. de M. Prego, A. A. de Lima, M. L. de Campos, E. A. da Silva, and S. L. Netto (2021, 2). Fault detection and classification in oil wells and production/service lines using random forest. Journal of Petroleum Science and Engineering 197, 107879. Pevný, T. (2016, 2). Loda: Lightweight on-line detector of anomalies. Machine Learning 102(2), 275–304. Shyu, M.-L., S.-C. Chen, K. Sarinnapakorn, and L. Chang (2003). A Novel Anomaly Detection Scheme Based on Principal Component Classifier. In IEEE Foundations and New Directions of Data Mining Workshop, in conjunction with the Third IEEE International Conference on Data Mining (ICDM’03), Volume 9, pp. 1–9. Turan, E. M. and J. Jaschke (2021, 6). Classification of undesirable events in oil well operation. Proceedings of the 2021 23rd International Conference on Process Control, PC 2021, 157–162. Vargas, R. E., C. J. Munaro, P. M. Ciarelli, and J. C. De Araujo (2017, 7). Proposal for two classifiers of offshore naturally flowing wells events using k-nearest neighbors, sliding windows and time multiscale. 2017 6th International Symposium on Advanced Control of Industrial Processes, AdCONIP 2017, 209–214. Vargas, R. E. V., C. J. Munaro, P. M. Ciarelli, A. G. Medeiros, B. G. d. Amaral, D. C. Barrionuevo, J. C. D. d. Araújo, J. L. Ribeiro, and L. P. Magalhães (2019, 10). A realistic and public dataset with rare undesirable real events in oil wells. Journal of Petroleum Science and Engineering 181, 106223.