Software Integration of Power System Measurement Devices with AI Capabilities

Academic Editor: Roberto Zivieri Received: 12 December 2024 Revised: 23 December 2024 Accepted: 25 December 2024 Published: 28 December 2024 Citation: Arenas-Ramos, V.; Cuesta, F.; Pallares-Lopez, V.; Santiago, I. Software Integration of Power System Measurement Devices with AI Capabilities. Appl. Sci. 2025,15, 170. https://doi.org/10.3390/ app15010170 Copyright: © 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/ licenses/by/4.0/). Article Software Integration of Power System Measurement Devices with AI Capabilities Victoria Arenas-Ramos 1,*,† , Federico Cuesta 2,*,† , Victor Pallares-Lopez 1and Isabel Santiago 1 1 Departamento de Ingeniería Electrónica y de Computadores, Campus de Rabanales, Universidad de Córdoba, 14071 Córdoba, Spain; vpallar[email protected] (V.P.-L.); [email protected] (I.S.) 2Escuela Técnica Superior de Ingeniería, Universidad de Sevilla, Camino Descubrimientos, E-41092 Sevilla, Spain *Correspondence: [email protected] (V.A.-R.); [email protected] (F.C.) †These authors contributed equally to this work. Featured Application: Distributed Power Systems Monitoring. Abstract: The latest changes on the distribution network due to the presence of distributed energy resources (DERs) and electric vehicles make it necessary to monitor the grid using a real-time high-precision system. The present work centers on the development of an open-source software platform that allows for the joint management of, at least, power quality monitors (PQMs), phasor measurement units (PMUs), and smart meters (SMs), which are three of the most widespread devices on distribution networks. This framework could work remotely while allowing access to the measurements in a comfortable way for grid analysis, prediction, or control tasks. The platform must meet the requirements of synchronism and scalability needed when working with electrical monitoring devices while considering the large volumes of data that these devices generate. The framework has been experimentally validated in laboratory and field tests in two photovoltaic plants. Moreover, real-time Artificial Intelligence capabilities have been validated by implementing three Machine Learning classifiers (Neural Network, Decision Tree, and Random Forest) to distinguish between three different loads in real time. Keywords: power quality monitor; phasor measurement unit; smart meter; open-source software; distributed measurement system; photovoltaic plant; load identification; artificial intelligence; neural network; decision tree; random forest 1. Introduction In recent years, there has been a significant improvement in the devices available for monitoring electrical systems focused on different applications, in terms of their performance, variety, cost, availability communication capabilities, acquisition, or processing software. An interesting review of the different monitoring devices that appear in a smart grid can be found in [ 1 ], where the fundamental characteristics and applications of three of the most common ones are briefly described: a phasor measurement unit (PMU), a power quality equipment (PQM), and a smart meter (SM). This performance improvement has been enhanced, among others, by the notable increase in distributed energy resource (DER) environments, photovoltaic (PV) panels, and electric vehicle charging, which pose substantial challenges from a power quality point of view [ 2 , 3 ]. In this way, monitoring devices are enabling the development of various Appl. Sci. 2025,15, 170 https://doi.org/10.3390/app15010170 Appl. Sci. 2025,15, 170 2 of 27 applications at different levels. For example, they have been used in fault detection: using power quality monitoring data to estimate fault location on distribution feeders [ 4 ], islanding the location of a ground fault [ 5 ], automatic fault location, and high impedance faults location on distribution networks using PMUs [ 6 – 8 ]. Another major set of applications involves the estimation of the state of the distribution system [ 9 – 11 ]. In [ 12 ], a recent and comprehensive survey is presented on the estimation of the state of the power system using multiple data sources. Another area of interest involves the applications of real-time power flow monitoring and control in microgrids [ 13 ] or a stability study using renewable sources [ 14 ]. These previous applications are typically carried out using both PQMs and PMUs, while for Non-Intrusive Load monitoring (NILM), applications for distributed energy resources (DERs) [ 15 ], residential settings [ 16 ], or industrial settings [ 17 ] both SMs and PQMs are used. On a different note, the availability of large volumes of generated data has made it common to use Artificial Intelligence techniques for these tasks [ 18 ]: recent reviews of machine learning techniques in power system security and stability [19,20], fault diagnosis [21], and NILM applications [16,17,22] can be found. Despite the benefits, integrating these devices poses significant challenges. Each electrical monitoring device has unique characteristics and requirements, such as transmission speed and synchronization, that differentiate it from others. As a result, each device typically operates within its own specialized software. In addition, each manufacturer often provides proprietary software that is easy to use with their equipment but incompatible with others. This creates a highly heterogeneous environment where linking data from different sources is challenging. However, in grids where different electrical monitoring devices are used simultaneously, it is highly desirable to have all these variables unified in a single data stream. A large number of applications would benefit from handling joint data [23]. It is therefore of interest to implement an architecture based on open-source tools that allows and facilitates the integration of measurements from different devices, provides real-time access and interaction with the data, and includes the possibility of implementing systems based on Artificial Intelligence. Data must be stored synchronously, despite the varying levels of synchronism among the devices. Having synchronized measurements facilitates the correlation of measurements from different and distant equipment, even if the comparison is limited by the lower-quality equipment. Alternative software tools are available in these bibliography [ 24 – 27 ], although they have limitations in terms of flexibility and scalability when given the different applications for which they are intended. For example, in [ 24 ], trend data from the PQM are not taken into account; only event data registered from the PQM event recorder are considered. In [ 25 ], it was necessary to extract the stored data in the CSV/JSON format, which would be a huge drawback for any application related to Artificial Intelligence in real time due to the format slowness. In [ 26 ], a version is presented that improves the management of CSV/JSON packages, but the main problem remains. Finally, in [ 27 ], a solution is proposed where PQM and PMU data are admitted though the use of the Message Queuing Telemetry Transport (MQTT); therein, cloud computing significantly slows down the data flow, which has led to the conclusion that this solution requires further optimization. In this context, this work presents and implements a new alternative developed using widely recognized open-source tools for integrating heterogeneous electrical monitoring devices. This approach aims to mitigate the abovementioned issues while facilitating the application of machine learning techniques. In addition, several experimental tests are conducted to demonstrate the system’s capabilities. This article continues in Section 2with a description of the most commonly used electrical monitoring devices. Following this, Section 3presents the integration framework Appl. Sci. 2025,15, 170 3 of 27 we developed. Section 4then presents two experimental test cases. Subsequently, Section 5 demonstrates the integration of the proposed tool with real-time machine learning-based decision systems. Finally, the article ends with a conclusion section. 2. Monitoring Devices As mentioned in the Introduction, there is a wide range of electrical monitoring devices on the market, each with different characteristics and performances. The quality of the monitoring system will depend on both the equipment it is composed of and its distribution over the grid. High-quality equipment will return more accurate and complete information than other low-end devices. Furthermore, depending on the characteristics of each device, it may be advisable to place it closer to or further away from the consumers or producers [3,28–30] . For DERs such as PV plants, the placement and quality of the monitoring devices are crucial for real-time monitoring of energy production, consumption, and storage, enabling precise control and optimization to maintain grid stability and efficiency. Figure 1shows an example of a typical distribution network, where PQMs are placed near the consumers, with PMUs at the beginning of each line and an SM at each consumer. This section explains what this equipment consists of and the reasons that justify its placement. Figure 1. Example of a distribution network with meassurement devices. 2.1. Power Quality Monitors (PQMs) and Power Quality Analyzers (PQAs) The IEEE Std C37.2-2008 standard [ 31 ] defines a PQM as a device capable of monitoring the electrical parameters used in power quality (PQ) analysis, specifically those parameters that determine compliance or non-compliance with supply quality regulations. These parameters include RMS voltage and current values, frequency values, power, phase unbalance, an harmonics and interharmonics in an electrical signal. Along with these, parameters of other nature may also appear. An “extended” version of this type of equipment is the Power Quality Analyzer (PQA), which further processes the data, checking whether the measurements comply with the standards or not. PQMs and PQAs usually also include an event recorder apart from their trend data. The difference between a PQM and a PQA is subtle; however, a PQA can be regarded as an enhanced version of a PQM. In general, the particular parameters they measure and the way they are treated will strongly depend on the objectives when installing a meter. All measurements collected by a PQM are derived from those collected by Current Transformer (CT) and Power Transformer (PT) modules, along with additional measurements that require further processing, such as harmonic measurements and Total Harmonic Appl. Sci. 2025,15, 170 4 of 27 Distortion (THD) calculation. In three-phase devices, most measurements are presented three times. Moreover, these measurements are often shown with different time aggregations, significantly increasing the amount of available data. As a result, the volume of trend data can be quite substantial. There is no regulation or standard that indicates which parameters should be chosen, although IEEE Std 1159-2019 [ 32 ] gives some recommendations. 2.2. Phasor Measurement Unit (PMU) and µPMU A PMU is a device developed with the purpose of measuring voltage and current phasors, frequency, and the Rate Of Change Of Frequency (ROCOF) synchronized with an absolute time reference through a Global Positioning System (GPS). Precisely, one of the main features of PMUs is their high sampling rate and time accuracy. For a grid frequency of 50Hz, sample frequencies of 10, 25, or 50 Frames per Second (FPS) are possible. This implies measuring or registering data every 100 ms, 50 ms, or 20 ms, respectively. Time synchronization ensures precise alignment of the data, allowing for an effective comparison of measurements taken at different points on the grid. This opens up the option for more efficient management, enhancing problem detection and increasing the ability to address issues. This is why PMUs are widely used in both the transmission and distribution stages of the power system [ 11 , 12 ]. Their accuracy is also significant, reaching ± 1% in magnitude and ±1◦in phase accuracy. In recent years, there has been a large growth in the use of DERs. These systems introduce considerable complexities and effects in the grid that require thorough study. While PMUs have been effective in monitoring, this increase in complexity calls for a technology that can detect irregularities with even greater accuracy and higher resolution than conventional PMUs. That is why the so-called µ PMUs [ 33 ] were developed. These devices reach rates of 100phasors/s for a signal of 50Hz and 120 phasors/s for 60Hz, meaning they return instantaneous grid values every half cycle. They also exceed the measurement accuracy given by PMUs, achieving angular accuracy values of up to ± 0.010 ◦ and amplitude accuracy values of up to ±0.05%. They are otherwise similar equipment. 2.3. Smart Meter A smart meter calculates consumption or production in a more detailed way than conventional meters. Domestic meters in urban low-voltage networks are common. They allow local service providers to monitor and manage the consumption of each user effectively. Their accuracy ranges between ± 1% and ± 2%. They usually have an adjustable sending rate between 1 and 60 min. 2.4. Applications and Characteristic Comparison Each of these devices has a common use in at least one specific application. However, often both the PMU and the PQM are devices that, due to their characteristics and measurement accuracy, lend themselves to a large number of applications. SMs are more dedicated to their application of regulating power consumption, but precisely for this reason, they are also very common. Table 1shows a large number of common applications found in the literature that can be covered by these devices to a greater or lesser extent. In addition, Table 2shows a comparison of the communications characteristics of the three electrical monitoring devices. In [ 1 , 34 ], there are more in-depth reviews of both the equipment and their applications. Appl. Sci. 2025,15, 170 5 of 27 Table 1. Summary of applications for µPMU, PQM and SM. Application µPMU PQM 1Smart Meter Monitoring Yes [14]Yes: designed for this purpose [4,13]Yes [35,36] Harmonic Analysis No Yes [37] No Event Detection Yes [38–40]Most devices include an event recorder No Protection and/or Control Yes: designed for this purpose [41,42]Less recommended No Fault Location Yes [6–8] Yes [4,5] Yes [43] Modeling State estimation [9,10,12]Load modeling [2,44]Load modeling [45,46] Topology Identification Yes [47,48] No No Demand Forecasting and Management Less recommended Yes [49] Yes [50–52] 1Only trend data is considered; event data is excluded. Table 2. µPMU, PQM, and SM communication capability comparison. Feature µPMU PQM Smart Meter Sampling Rate 25,600 m/s at 50 Hz 25,600 m/s at 50 Hz from 1600 m/s up to 51,200 m/s Transmission Rate from 10 ms up to 100 ms from 1 s up to 10 min from 1 s up to 10 min Transmission Speed Crucial. IEEE Std C37.118.2 [53]Not crucial Not crucial Measures Voltage and current phasors. Frequency. ROCOF PQ data. Frequency, Voltage, Current, power, THD, TDD, harmonics, etc. Power and energy data. Frequency, voltage, current, etc. Data Volume Very high High Low Synchronism Very important. GPS. Less important. NTP (usually) Less important Most Common Protocol IEEE Std. C37.118.2. Modbus ASCII Modbus RTU 3. Proposed Integration Framework 3.1. Requirements The aim of this work is to create a software architecture that can accommodate the measurements of different electrical monitoring devices while making the data available to any artificial intelligence task in real time. Particular attention is given to PMUs, as they exhibit distinctly different behavior compared to the other equipment discussed in the previous section (see Table 2). On the contrary, PQMs and SMs shows some similarities, as both devices collect power and energy data. Usually, PQMs include a wider range of measures, including more detailed power quality data. However, the specific list of measurements may vary based on the manufacturer. They both rely on standard communications protocols on the market. The case of PMUs is different. For them, sample synchronization, efficient storage, and high transmission speed are essential. Since PMUs are capable of sending up to 100 FPS, very high data volumes can be reached in a short period of time. This requires an Appl. Sci. 2025,15, 170 6 of 27 infrastructure that can not only store a large volume of data but also access it efficiently. The infrastructure must also be able to respect the original time stamping of the PMU. PMUs generally use the IEEE C37.118.2 standard [ 53 ]. Within this standard, the need for a Phasor Data Concentrator (PDC), a software that collects and manages the phasors, is specified. This article proposes a valid framework for PMUs, SMs, and PQMs that collect data based on commercial protocols while incorporating the standards of the IEEE Std. C37.118.2 and the use of a PDC. Firstly, in Section 3.2, a brief sketch of the design is shown. Then, in Section 3.3, the tools chosen are explained, together with some alternatives of similar performance. The objectives that are therefore set for the architecture as a whole, and which must therefore be met by each and every one of the tools used, are as follows: 1. Integration capability: Since the main goal of storing data jointly is its comparison and correlation, the system should be integrable with a wide range of protocols. 2. Scalability: The ability to manage a large volume of data. 3. Easy user access: Queries should be user-friendly and efficient in terms of time. Data must be accessible for both analysis and real-time monitoring. 4. Open Source: It aims to embrace the open-source philosophy, making the system accessible to all users without reliance on proprietary software. 5. Real-Time capability: The system must provide real-time monitoring capabilities. Additionally, in order to support AI-driven applications, it should be able to execute machine learning processes in real time. 3.2. System Architecture Figure 2shows the sketch of the framework proposed in this work. The system features a modular design, with each tool required to meet the specifications outlined in Section 3.1. Firstly, the electrical monitoring devices are represented. They can be either PMUs using the IEEE C37.118.2 standard [ 53 ] or other devices such as PQMs or SMs using generic communications protocols (Modbus, SMTP, MQTT, etc.). An agent is required for each protocol implemented in the system, as well as a PDC to collect the measurements from the PMUs. Multiple PDCs or agents can be added as needed based on the equipment’s distribution across the grid. Figure 2. Proposed infrastructure. Next in the data flow is the database. It was designed to be the core component of the system architecture, differing from systems like the FIWARE platform where a data Appl. Sci. 2025,15, 170 7 of 27 distribution agent is central. This approach is common in tools where data storage is critical [ 25 , 27 ], helping to avoid unnecessary data streams. However, it requires a database that can integrate with multiple agents and PDCs. With numerous database management systems available, it is crucial to select the one that best meets our needs. Relational databases (SQL) are useful for avoiding data duplication but are slow with large data volumes, making them unsuitable for PMU measurements. In contrast, non-relational databases (NoSQL) are designed to handle large data volumes efficiently and offer faster data handling speeds. These databases are often specialized for specific data types and optimized accordingly. Among NoSQL types, time series database managers (TSDBs) stand out by storing data as value–timestamp pairs, making them more efficient than other database types for time series data [54,55]. Finally, back to the framework requirements, this architecture must be user-friendly. In this regard, it was considered essential to allow real-time access to the stored data, both for monitoring and real-time analysis, allowing the incorporation of algorithms that feed back the generated data. Likewise, it will be necessary to be able to access the data offline if a subsequent analysis is required. The software used for this purpose would depend on the client, but the TSDB will need to be widely compatible for better performance. 3.3. Proposed Software Integration Tools and Alternatives Based on the sketch drawn in Section 3.2, a series of open-source tools were selected, as shown in Figure 3. Each of the indicated tools could be replaced by a similar one as long as they meet the requirements described in Section 3.1. The tools used, along with similar performance alternatives, are described below. Figure 3. Proposed framework with each chosen software tool. InfluxDB version 2.0 was chosen as the TSDB, which is an open-source TSDB developed by the InfluxData company. It offers a comprehensive ecosystem, including the Telegraf agent [ 56 ], which is capable of receiving data from an enormous number of protocols. Another widely used open-source TSDB is MongoDB, although it does not have its own agent that is as versatile as Telegraf, so it would be necessary to resort to another type of middleware to integrate messages, such as Node-RED.InfluxDB, which appears to be the most suitable option due to its versatility [54]. Appl. Sci. 2025,15, 170 8 of 27 The chosen PDC is OpenPDC, which is an open-source program widely used in the literature [ 24 , 25 , 57 ]. This data concentrator has the capability to forward the information that reaches it to other PDCs through the Streaming Telemetry Transport Protocol (STTP IEEE 2664) [58]. This protocol, still under development by the IEEE, has been optimized to exchange time series data. Due to the protocol’s novelty, it has not yet been implemented in the TSDB. To address this, a middleware called STTP2HIDS, developed by the Grid Protection Alliance (GPA), was used to convert the data into write requests for InfluxDB. There are other proprietary PDCs, such as the enhanced Phasor Data Concentrator (ePDC) developed by Electric Power Group (EPG), or the VCL-PDC by Valiant Communications. Nevertheless, OpenPDC is the only open-source PDC that has been widely validated. All of them have the same problem when transmitting data to a generic TSDB, so STTP2HIDS would have to be used anyway. In [ 24 ], OpenHistorian, another GPA product, was used as the middleware; however, it is much more bulky and cumbersome for this purpose than STTP2HIDS. Lastly, having such a popular TSDB makes it very easy to access the data. Using InfluxDB, it is possible to download the data in the CSV format to analyze them externally offline with any software, such as GNU Octave 9.3 or Python 3.11 , or it is possible to access the database online with an API capable of performing periodic queries. In this case, a Python API was used. It allows data to be monitored and processed in real time. Python’s versatility enables the use of well-known tools such as TensorFlow 2 or Keras 3 to perform Artificial Intelligence tasks directly on the data. Likewise, it will be possible to monitor the data in real time with a monitoring software such as Grafana 11.4. 4. Laboratory and Field Tests One of the main advantages of this platform is its ability to store and compare measurements of various types, enabling them to complement each other. To this end, a series of tests were carried out both in the laboratory and on a distribution grid using DERs, with both commercial devices and self-designed equipment. A Grafana dashboard was designed, although the data were analyzed offline by downloading them in the CSV format. 4.1. Laboratory Testbed and Results The designed software was implemented and tested in the laboratory using four units of the most demanding commercial electrical monitoring devices. Two Powerside PQMs, specifically the three-phase PQube3 model and two µ PMUs of the same brand, were used as measuring equipment. The µ PMUs generate the measurements shown in Section 2.2 for each phase, plus some power measurements that are only displayed on the device and are not transmitted. The PQM, on the other hand, generates more than 3000 values among power, energy, harmonics, and other PQ measurements. As described in Section 3.1, µ PMU uses, as usual, the IEEE Std. C37.118.2. It can be configured to the different existing versions, and in this case, the 2011 version was picked. In the case of PQMs, they use Modbus TCP/IP. The four devices were connected simultaneously to the TSDB, storing the data they generate in real time. Both the TSDB and the Telegraf agent were located on one PC, while OpenPDC ran on another PC, both within the same local network. Grafana monitoring software was also in the same computer where the TSDB was hosted. Figure 4 shows how the equipment was placed in the laboratory, while Figure 5shows the data flow. Figure 6shows a dashboard in Grafana where the most significant measurements collected by the PQMs are represented in real time. In the upper-left corner, the PQM for which the data are to be displayed is selected, whereas in the upper-right corner, the time and date are selected. A similar dashboard was created for the µPMUs. Appl. Sci. 2025,15, 170 9 of 27 Figure 4. Experimental testbed in the laboratory. Figure 5. Experimental test data flow. Figure 6. Real-time monitoring of a PQM (PQube3) analyzer with Grafana dashboard. In these tests, the equipment was conveniently compared in pairs. First, we compared the equipment of the same type with each other, followed by comparisons between the Appl. Sci. 2025,15, 170 16 of 27 four categories according to the LED lamp connected to the line at any given time: “None”, “Lamp1”, “Lamp2”, or “Lamp3”. As is usual in this type of technique, the data must be subjected to preprocessing before being used. Specifically, a feature selection was performed, initially excluding harmonics that were too low and going from 39 to 21 harmonics. In addition, a scaling of the type MaxAbsScaler was applied so that each feature would be scaled according to its maximum absolute value, and the value of the feature for training would be in the range [ 0.0,1.0 ] . Figure 16, for illustrative purposes, shows the average normalized values of the 21 harmonics considered. 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 Harmonic 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Voltage (V) Mean Voltage Harmonics Lamp1 Lamp2 Lamp3 Figure 15. Average voltage harmonics (2nd to 40th) on each of the three lamps. The data were then appropriately divided into a training subset and a test subset using the train_test_split function from the Scikit_learn library, which randomly selects values to ensure a balanced and correct distribution of data from each of the categories in both subsets. Proportions of 80% of training data and 20% for validation were used. This resulted in a subset of scaled training data, with approximately 1600 entries for each category, and a test subset, with approximately 400 values for each class. 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 Harmonic 0.0 0.2 0.4 0.6 0.8 1.0 Normalized voltage Mean Normalized Voltage Harmonics Lamp1 Lamp2 Lamp3 Figure 16. Normalized values of the considered average voltage harmonics on each of the three lamps. The first classifier implemented was a Neural Network. Although there is a wide variety of advanced neural networks suitable for these tasks (including RBF, CNN, BRNN, Appl. Sci. 2025,15, 170 17 of 27 KNN, DNN, GMDH, etc.) [ 17 ], for the sake of implementation and comparison with the other two methods, the classic Multi-layer Perceptron classifier structure available in Scikit-learn has been used, which allows for supervised learning by optimizing the log loss function. In this case, the input layer will be formed by 21 nodes and the output layer by 4 nodes (one for each of the considered categories). Configuring the NN requires optimizing the cross-tuning of a significant number of hyperparameters, which influence the accuracy of the network, as can be seen in Figure 17. To assist in this task, Scikitlearn implements the GridSearchCV function, which performs an exhaustive search over specified parameter values optimized by cross-validated grid search over a parameter grid. In this way, optimized hyperparameters were established with a configuration of two internal layers of 5 and 10 neurons, respectively, as well as through the use of a stochastic gradient descent (SGD) optimizer, with an adaptive learning rate and an activation function for the hidden layer of the hyperbolic tan type: f(x) = ex−e−x ex+e−x(2) (a) Hyperbolic tan activation function. (b) Sigmoid activation function. Figure 17. Number of neurons in the two hidden layers vs. accuracy with different activation functions. As a result of training this model, a predictor with 100% accuracy was obtained in both the training and validation subsets. Figure 18 shows the evolution of the loss function during training and the confusion matrix obtained with the test data subset. In general, classifiers based on neural networks tend to exhibit strong generalization capabilities. However, sometimes, their black-box nature makes it challenging to extract information about the criteria used to classify an input value into specific categories. In addition, the computation time can be significant if a large number of hidden layers and neurons are used. In such situations, it may be appropriate to use a non-parametric classifier such as a Decision Tree. Appl. Sci. 2025,15, 170 18 of 27 0 50 100 150 200 250 300 Iterations 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 Cost Loss Curve (a) Loss curve None Lamp 1 Lamp 2 Lamp 3 Predicted None Lamp 1 Lamp 2 Lamp 3 Observed 414 0 0 0 0400 0 0 0 0 417 0 0 0 0 403 Confusion Matrix NN 0 50 100 150 200 250 300 350 400 (b) Confusion matrix of NN on test data. Figure 18. Multi-Layer Perceptron training and test results. The goal of the DT is to construct a binary tree structure that efficiently classifies input values through a progressive features evaluation process until reaching a terminal node (leaf) that corresponds to the category associated to the input value. In this case, the input value would be the recorded harmonics. The conditions are established based on the values of the harmonics, and the result should be a leaf corresponding to each of the four categories. DT training consists of optimizing the decision rules at each node, based on input data, to achieve better divisions. This process often utilizes metrics such as the Gini Index to guide the optimization: Gini =1− n ∑ i=1 P2 i(3) where Pi is the probability that an element is classified in a given category. Thus, the Gini Index, which ranges from 0 to 1, determines the impurity of a set, indicating how mixed or heterogeneous the set is. Therefore, using the Scikit-learn library, a DT was trained to select the best split at each node, using the Gini Index, and with a maximum depth of 3 to reduce computational time during evaluation. The DT obtained achieved 100% accuracy for both the training and test subsets. Figure 19 shows the structure of the Decision Tree and the confusion matrix when evaluating the test dataset. As can be seen in Figure 19a the tree managed to establish four leaves with purity ( Gini = 0) corresponding to each of the four categories using a classification method based on a progressive binary division of the scaled values of harmonics H29, H5, and H27. The effectiveness of this classification approach becomes evident when examining the distribution of these harmonic values within the available dataset, as shown in Figure 20. This results in a classifier that enables rapid classification, though it may be more sensitive to the values of specific harmonics and thus potentially less robust compared to the NN approach. To mitigate this issue, it is possible to employ a classifier that utilizes a set of DTs, known as Random Forest, which combines the results from all of these trees. Although it is possible to use a large number of trees, in this case, keeping in mind the real-time implementation, only four trees were calculated. Each of them was trained on a subset of the training data to facilitate the search of diverse classifications. As a result, an RF with an accuracy of 100% on the training and test subsets was obtained, which was composed of the four DTs shown in Figure 21. As can be observed, the decisions were made based on the values of nine harmonics (H3, H5, H9, H16, H23, H27, H33, H34, and H36), thereby increasing robustness with respect to the DT. Appl. Sci. 2025,15, 170 19 of 27 The performance of these three classifiers applied in real time will be shown in the next section. gini = 0.0 samples = 1639 value = [0, 0, 0, 16 39] class = Lamp 3 gini = 0.0 samples = 1625 value = [0, 0, 1625 , 0] class = Lamp 2 gini = 0.0 samples = 1629 value = [1629, 0, 0 , 0] class = None H27 <= 0.217 gini = 0.5 samples = 3264 value = [0.0, 0.0, 1 625.0, 1639.0] class = Lamp 3 H5 <= 0.332 gini = 0.667 samples = 4893 value = [1629, 0, 1 625, 1639] class = Lamp 3 True gini = 0.0 samples = 1642 value = [0, 1642, 0 , 0] class = Lamp 1 False H29 <= 0.591 gini = 0.75 samples = 6535 value = [1629, 164 2, 1625, 1639] class = Lamp 1 (a) Decision Tree. None Lamp 1 Lamp 2 Lamp 3 Predicted None Lamp 1 Lamp 2 Lamp 3 Observed 414 0 0 0 0400 0 0 0 0 417 0 0 0 0 403 Confusion Matrix DT 0 50 100 150 200 250 300 350 400 (b) Confusion matrix of DT on test data. Figure 19. Decision Tree structure and test results. Figure 20. Harmonic values distributions for each class (None, Lamp1, Lamp2, Lamp3) in training dataset. Appl. Sci. 2025,15, 170 20 of 27 gini = 0.0 samples = 389 value = [449, 0, 0, 0] class = None gini = 0.004 samples = 456 value = [0, 0, 1, 527] class = Lamp 3 H3 <= 0.249 gini = 0.498 samples = 845 value = [449, 0, 1, 527] class = Lamp 3 gini = 0.0 samples = 445 value = [0, 0, 506, 0] class = Lamp 2 H23 <= 0.61 gini = 0.665 samples = 1290 value = [449, 0, 507, 527] class = Lamp 3 True gini = 0.0 samples = 450 value = [0, 517, 0, 0] class = Lamp 1 False H27 <= 0.56 gini = 0.749 samples = 1740 value = [449, 517, 507, 527] class = Lamp 3 gini = 0.0 samples = 434 value = [0, 0, 519, 0] class = Lamp 2 gini = 0.0 samples = 422 value = [0, 478, 0, 0] class = Lamp 1 gini = 0.0 samples = 437 value = [0, 0, 0, 502] class = Lamp 3 H27 <= 0.534 gini = 0.499 samples = 856 value = [0, 478, 519, 0] class = Lamp 2 gini = 0.0 samples = 432 value = [501, 0, 0, 0] class = None True H27 <= 0.213 gini = 0.666 samples = 1293 value = [0.0, 478.0, 519.0, 502.0] class = Lamp 2 False H5 <= 0.242 gini = 0.75 samples = 1725 value = [501, 478, 519, 502] class = Lamp 2 gini = 0.0 samples = 414 value = [486, 0, 0, 0] class = None gini = 0.0 samples = 413 value = [0, 0, 0, 483] class = Lamp 3 gini = 0.0 samples = 452 value = [0, 526, 0, 0] class = Lamp 1 gini = 0.0 samples = 435 value = [0, 0, 505, 0] class = Lamp 2 H9 <= 0.038 gini = 0.5 samples = 827 value = [486, 0, 0, 483] class = None True H5 <= 0.769 gini = 0.5 samples = 887 value = [0, 526, 505, 0] class = Lamp 1 False H33 <= 0.391 gini = 0.75 samples = 1714 value = [486, 526, 505, 483] class = Lamp 1 gini = 0.0 samples = 415 value = [0, 0, 0, 480] class = Lamp 3 gini = 0.375 samples = 15 value = [0, 0, 4, 12] class = Lamp 3 gini = 0.268 samples = 431 value = [0, 80, 423, 0] class = Lamp 2 gini = 0.259 samples = 451 value = [0, 440, 78, 1] class = Lamp 1 H36 <= 0.214 gini = 0.016 samples = 430 value = [0, 0, 4, 492] class = Lamp 3 H34 <= 0.276 gini = 0.501 samples = 882 value = [0, 520, 501, 1] class = Lamp 1 gini = 0.0 samples = 423 value = [482, 0, 0, 0] class = None True H16 <= 0.413 gini = 0.667 samples = 1312 value = [0, 520, 505, 493] class = Lamp 1 False H9 <= 0.046 gini = 0.75 samples = 1735 value = [482, 520, 505, 493] class = Lamp 1 Figure 21. Random Forest composed of 4 DTs. 5.2. Online Experimental Results To validate the performance of the classifiers and to achieve more accurate comparisons, the PQM’s capability to record information from three phases simultaneously was taken advantage of. Consequently, one lamp was connected to each of the three phases. This allowed for the three classifiers (previously trained) to be evaluated simultaneously on the three lamps with the same on and off times. To accomplish this, starting from a state with lamps off, they were turned on and off multiple times, with varying durations as illustrated in Figure 22, for a period of approximately 55 min. Figure 22. Experimental results: Lamp connected at each phase vs. predictions. Appl. Sci. 2025,15, 170 21 of 27 The data were sampled and evaluated every second by applying the three classifiers to each of the phases. As can be seen in Figure 22, both the NN and the RF predicted satisfactorily most of the cases. Only in two instances during lamp 3 ignition was it estimated as lamp 2, possibly due to a phenomenon of Inrush Current [ 60 ]. In contrast, the DT exhibited additional errors in the initial switch-on and switch-off of lamp 1, estimating it as lamp 2, possibly due to its lower robustness to Inrush Current or temperature phenomena [61]. Figure 23 shows the confusion matrix of each algorithm on the experimental results. Table 5shows in detail the performance results obtained during the experiment with each of the models. The standard metrics were calculated based on the number of True Positives (TPs), False Positives (FPs), False Negatives (FNs) and True Negatives (TNs), which are defined as Accuracy =TP +TN TP +FP +TN +FN (4) Precision =TP TP +FP (5) Recall =TP TP +FN (6) F1-Score =2×Precision ×Recall Precision +Recall (7) MCC =(TP ×TN)−(FP ×FN) p(TP +FP)×(TP +FN)×(TN +FP)×(TN +FN)(8) Additionally, to understand the results of the DT, it is useful to compare the harmonics measured during the real-time test with those available in the learning phase. As shown in Figure 24, there are differences between the harmonics used in the learning phase (see Figure 20) and those measured during the experimental phase. The DT proved to be more sensitive to these differences, while both the NN and the RF demonstrated their capacity for generalization. Furthermore, the voltage harmonics of lamp 3 corresponding to the only two instances in which the NN and RT failed can be clearly seen as outliers in Figure 24. As can be seen, both the NN and the RF obtained the same result, with practically no failures. However, the NN required a computation time of 5.5 times that of the DT, while the RF required only 1.67 times that time. None Lamp 1 Lamp 2 Lamp 3 Predicted None Lamp 1 Lamp 2 Lamp 3 Observed 6825 000 0 1071 0 0 0 0 1071 0 0 0 2 1069 Confusion Matrix NN (Experiment) 0 1000 2000 3000 4000 5000 6000 (a) Neural Network. None Lamp 1 Lamp 2 Lamp 3 Predicted None Lamp 1 Lamp 2 Lamp 3 Observed 6825 000 0 956 115 0 0 0 1071 0 0 0 2 1069 Confusion Matrix DT (Experiment) 0 1000 2000 3000 4000 5000 6000 (b) Decision Tree. None Lamp 1 Lamp 2 Lamp 3 Predicted None Lamp 1 Lamp 2 Lamp 3 Observed 6825 000 0 1071 0 0 0 0 1071 0 0 0 2 1069 Confusion Matrix RF (Experiment) 0 1000 2000 3000 4000 5000 6000 (c) Random Forest. Figure 23. Confusion matrix of the experimental results. Appl. Sci. 2025,15, 170 22 of 27 Table 5. Metrics of the experimental results. Class Model Support TP FN FP TN Accuracy Precision Recall F1-Score MCC NN 6825 6825 0 0 3213 1.00000 1.00000 1.00000 1.00000 1.00000 None DT 6825 6825 0 0 3213 1.00000 1.00000 1.00000 1.00000 1.00000 RF 6825 6825 0 0 3213 1.00000 1.00000 1.00000 1.00000 1.00000 NN 1071 1071 0 0 8967 1.00000 1.00000 1.00000 1.00000 1.00000 Lamp1 DT 1071 956 115 0 8967 0.98854 1.00000 0.89262 0.94327 0.93879 RF 1071 1071 0 0 8967 1.00000 1.00000 1.00000 1.00000 1.00000 NN 1071 1071 0 2 8965 0.99980 0.99814 1.00000 0.99907 0.99896 Lamp2 DT 1071 1071 0 117 8850 0.98834 0.90152 1.00000 0.94821 0.94327 RF 1071 1071 0 2 8965 0.99980 0.99814 1.00000 0.99907 0.99896 NN 1071 1069 2 0 8967 0.99980 1.00000 0.99813 0.99907 0.99895 Lamp3 DT 1071 1069 2 0 8967 0.99980 1.00000 0.99813 0.99907 0.99895 RF 1071 1069 2 0 8967 0.99980 1.00000 0.99813 0.99907 0.99895 Figure 24. Harmonic values distributions in training set (None, Lamp1, Lamp2, Lamp3) vs. real time experiment (NoneRT, Lamp1RT, Lamp2RT, Lamp3RT). 6. Conclusions The main novelty of this paper is its designed framework capable of integrating electrical monitoring devices, such as PMUs, PQMs, and SMs, as well as adding AI capabilities to the recorded data. This framework covers a necessity given the large number of applications based on electrical monitoring data and the growth of DERs in distribution grids, which increase the necessity of monitoring and forecasting in these grids. In addition, only open-source software has been used, which is an advantage in terms of transparency, flexibility, and customization. Moreover, the framework has been proven to be capable of executing real-time machine learning models, which is something not usually discussed in the literature for similar frameworks. This capability allows the framework to process and analyze electrical monitoring data more effectively, improving applications such as fault diagnosis, state estimation, and event detection. In general, the chosen scheme was tested with remarkable success, both in the laboratory and in a distribution grid. Appl. Sci. 2025,15, 170 23 of 27 Integration capabilities were tested in both laboratory and field tests. In the laboratory, it was observed that measurements were synchronized among devices of the same type, but there was a noticeable delay of about one second between different devices. This is likely caused by the different synchronization methods used: PMUs are synchronized via a GPS, while PQMs use an NTP. In the field test, a more complex scenario was created using configurable National Instruments equipment. All equipment was connected via TSN, enabling the PQMs to use the same GPS signal as the PMUs. In this test, the measurements were strongly synchronized, and the delay was less than one second. Ongoing efforts are focused on further characterizing this field environment. In conclusion, performance may vary depending on the measurement equipment used, although the framework adapts to even the most demanding environments. Due to the robustness of the software, no data loss was observed. Once data were properly registered, machine learning techniques were integrated in a real-time load identification example. Neural Network, Decision Tree, and Random Forest classifiers for real-time load identification were implemented and experimentally validated to identify the connected LED from the voltage harmonics recorded by a PQM. Of the three options, the Decision Tree was found to be the most sensitive to data variability, while both the Neural Network and the Random Forest had near 100% accuracy. In all three cases, real-time executions were performed without visible delay or data congestion. Even the Neural Network, which required more computing time, was executed seamlessly. This guarantees the integration of machine learning techniques with the framework, although the capacity to execute more complex classifiers would depend on other external factors, such as the processing power. With both integration and load identification tests, the capabilities of the framework were tested. Looking ahead, the framework has the potential to support a wide range of applications involving electrical devices and AI, including monitoring and control, state estimation, event detection, and fault location. Although certain software tools, such as the selected TSDB InfluxDB, could be substituted with similar alternatives, the tools utilized in this study delivered outstanding performance and notable success. Author Contributions: Conceptualization, V.A.-R. and F.C.; methodology, V.A.-R. and F.C.; software, V.A.-R. and F.C.; validation, V.A.-R., F.C. and V.P.-L.; formal analysis, F.C.; investigation, V.A.-R. and F.C.; resources, V.A.-R., F.C., V.P.-L., and I.S.; data curation, F.C.; writing—original draft preparation, V.A.-R. and F.C.; writing—review and editing, V.A.-R., F.C., and I.S.; visualization, V.A.-R. and F.C.; funding acquisition, F.C. and V.P.-L. All authors have read and agreed to the published version of the manuscript. Funding: This work has been supported by the Agencia Estatal de Investigación (AEI)-Spain, under grant PID2019-109071RB-I00, and the Spanish research subproject Monitoring And Integration of energy data with Seamless Temporal Accuracy for photovoltaic plants (MISTA) of the coordinated project Strategies for Aggregated Generation of PhotoVoltaic plants (SAGPV). Ref. PID2019-108953RAC22 and PID2019-108953RB-C21. Data Availability Statement: The original data presented in the study are openly available in the Helvia repository at http://hdl.handle.net/10396/30154 (accessed on December 2024 ). Acknowledgments: We give our gratitude to Solar del Valle, owner of the Cruz del Doctor and Cabeza Oliva PV plants, for their support in the development of this research. Conflicts of Interest: The authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results. Appl. Sci. 2025,15, 170 24 of 27 Abbreviations The following abbreviations are used in this manuscript: AI Artificial Intelligence BRNN Bidirectional Recurrent Neural Network CNN Convolutional Neural Network CT Current Transformer DER Distributed Energy Resource DNN Deep Neural Network DT Decision Tree FPS Frames per Second GMDH Group Method of Data Handling GPS Global Positioning System KNN K-Nearest Neighbor LAN Local Area Network NILM Non-Intrusive Load Monitoring NN Neural Network NoSQL Non-Relational Databases NTP Network Time Protocol PDC Phasor Data Concentrator PMU Phasor Measurement Unit µPMU Micro-Phasor Measurement Unit PQ Power Quality PQA Power Quality Analyzer PQM Power Quality Monitor PT Power Transformer PV Photo-Voltaic RF Random Forest RBF Radial Basis Function ROCOF Rate Of Change Of Frequency SGD Stochastic Gradient Descent SM Smart Meter SQL Structured Query Language STTP Streaming Telemetry Transport Protocol TC Transformation Center THD Total Harmonic Distortion TSDB Time Series Database TSN Time-Sensitive Networkig TVE Total Vector Error WAN Wide Area Network References 1. Saldaña-González, A.E.; Sumper, A.; Aragüés-Peñalba, M.; Smolnikar, M. Advanced Distribution Measurement Technologies and Data Applications for Smart Grids: A Review. Energies 2020,13, 3730. [CrossRef] 2. Leinakse, M.; Tani, P.; Kilter, J. Impact of Distributed Generation on Estimation of Exponential Load Models. IEEE Power Energy Soc. Gen. Meet. 2019,2019, 8974014. [CrossRef] 3. Yu, J.; Sun, Y.; Lu, J.; Zou, Q.; Xie, X.; Fu, B.; Liu, W.; Ye, L.; Chen, Y. Optimal placement of power quality monitors in complex distribution network based on topological degradation. In Proceedings of the 14th IEEE Conference on Industrial Electronics and Applications, ICIEA 2019, Xi’an, China, 19–21 June 2019; pp. 58–61. [CrossRef] 4. Kim, J.; Baran, M.E.; Lampley, G.C. Estimation of fault location on distribution feeders using PQ monitoring data. In Proceedings of the 2007 IEEE Power Engineering Society General Meeting, PES, Tampa, FL, USA, 24–28 June 2007. [CrossRef] 5. Selvaraj, G.; Thangaiyan, V.; Rajangam, K. Numerical method for islanding the location of ground fault in the material based distribution system. Mater. Today Proc. 2021,45, 634–639. [CrossRef] Appl. Sci. 2025,15, 170 25 of 27 6. Lee, J. Automatic Fault Location on Distribution Networks Using Synchronized Voltage Phasor Measurement Units. In Proceedings of the American Society of Mechanical Engineers, Power Division (Publication) POWER, Baltimore, MD, USA, 28–31 July 2014. [CrossRef] 7. Cui, Q.; Weng, Y. Enhance High Impedance Fault Detection and Location Accuracy via uPMUs. IEEE Trans. Smart Grid 2020,11, 797–809. [CrossRef] 8. Yablokov, A.; Ivanov, I.; Kulikov, F.; Tychkin, A.; Panaschatenko, A.; Umnov, Y. Investigation of fault location based on class M PMUs with simulated and real-field synchrophasors. Electr. Power Syst. Res. 2024,232, 110414. [CrossRef] 9. Chen, X.; Tseng, K.J.; Amaratunga, G. State estimation for distribution systems using micro-synchrophasors. In Proceedings of the Asia-Pacific Power and Energy Engineering Conference, APPEEC, Nanjing, China, 25–27 October 2016. [CrossRef] 10. Farajollahi, M.; Shahsavari, A.; Mohsenian-Rad, H. Tracking State Estimation in Distribution Networks Using Distributionlevel Synchrophasor Data. In Proceedings of the IEEE Power and Energy Society General Meeting, Portland, OR, USA, 5–10 August 2018. [CrossRef] 11. Dusabimana, E.; Yoon, S.G. A Survey on the Micro-Phasor Measurement Unit in Distribution Networks. Electronics 2020,9, 305. [CrossRef] 12. Cheng, G.; Lin, Y.; Abur, A.; Gomez-Exposito, A.; Wu, W. A Survey of Power System State Estimation Using Multiple Data Sources: PMUs, SCADA, AMI, and Beyond. IEEE Trans. Smart Grid 2024,15, 1129–1151. [CrossRef] 13. Artale, G.; Cataliotti, A.; Cosentino, V.; Cara, D.D.; Guaiana, S.; Panzavecchia, N.; Tine, G. Real-Time Power Flow Monitoring and Control System for Microgrids Integration in Islanded Scenarios. IEEE Trans. Ind. Appl. 2019,55, 7186–7197. [CrossRef] 14. Meegahapola, L.G.; Bu, S.; Wadduwage, D.P.; Chung, C.Y.; Yu, X. Review on Oscillatory Stability in Power Grids with Renewable Energy Sources: Monitoring, Analysis, and Control Using Synchrophasor Technology. IEEE Trans. Ind. Electron. 2021,68, 519–531. [CrossRef] 15. Jaramillo, A.F.M.; Laverty, D.M.; Morrow, D.J.; del Rincon, J.M.; Foley, A.M. Load modelling and non-intrusive load monitoring to integrate distributed energy resources in low and medium voltage networks. Renew. Energy 2021,179, 445–466. [CrossRef] 16. Ghosh, S.; Chatterjee, D. Non-intrusive identification of harmonic polluting loads in a smart residential system. Sustain. Energy, Grids Netw. 2021,26, 100446. : 10.1016/j.segan.2021.100446 [CrossRef] 17. Tanoni, G.; Principi, E.; Squartini, S. Non-Intrusive Load Monitoring in industrial settings: A systematic review. Renew. Sustain. Energy Rev. 2024,202, 114703. : 10.1016/j.rser.2024.114703 [CrossRef] 18. Stock, S.; Babazadeh, D.; Becker, C. Applications of Artificial Intelligence in Distribution Power System Operation. IEEE Access 2021,9, 150098–150119. [CrossRef] 19. Alimi, O.A.; Ouahada, K.; Abu-Mahfouz, A.M. A Review of Machine Learning Approaches to Power System Security and Stability. IEEE Access 2020,8, 113512–113531. [CrossRef] 20. Lal, M.D.; Varadarajan, R. A Review of Machine Learning Approaches in Synchrophasor Technology. IEEE Access 2023,11, 33520–33541. [CrossRef] 21. Vaish, R.; Dwivedi, U.; Tewari, S.; Tripathi, S. Machine learning applications in power system fault diagnosis: Research advancements and perspectives. Eng. Appl. Artif. Intell. 2021,106, 104504. : 10.1016/j.engappai.2021.104504 [CrossRef] 22. Chen, T.; Qin, H.; Li, X.; Wan, W.; Yan, W. A Non-Intrusive Load Monitoring Method Based on Feature Fusion and SE-ResNet. Electronics 2023,12, 1909. [CrossRef] 23. Constandache, N.; Stanescu, D.M.; Sanduleac, M.; Stanescu, C.; Tristiu, I.; Mandis, A. Smart meters, PMU and PQ data analysis in active distribution grids—Case studies in MV networks. In Proceedings of the 2018 International Conference on Applied and Theoretical Electricity, ICATE, Craiova, Romania, 4–6 October 2018. [CrossRef] 24. Castello, P.; Muscas, C.; Pegoraro, P.A.; Sulis, S.; Giannuzzi, G.M.; Pede, M.; Maiolini, C.; Pau, P.; Bassi, F.; Coluzzi, C. Integration of power quality and fault data into a PMU-based Wide Area Monitoring System. In Proceedings of the AMPS 2021—2021 11th IEEE International Workshop on Applied Measurements for Power Systems, Virtual Conference, 29 September–1 October 2021. [CrossRef] 25. Andersen, M.P.; Kumar, S.; Brooks, C.; Meier, A.V.; Culler, D.E. DISTIL: Design and implementation of a scalable synchrophasor data processing system. In Proceedings of the 2015 IEEE International Conference on Smart Grid Communications (SmartGridComm), Miami, FL, USA, 2–5 November 2016; Institute of Electrical and Electronics Engineers Inc.: Piscataway, NJ, USA, 2016; pp. 271–277. [CrossRef] 26. Kosen, I.; Huang, C.; Chen, Z.; Zhang, X.; Min, L.; Zhou, D.; Zhu, L.; Liu, Y. UPS: Unified PMU-Data storage system to enhance T+D PMU data usability. IEEE Trans. Smart Grid 2020,11, 739–748. [CrossRef] 27. Sodin, D.; Rudež, U.; Mihelin, M.; Smolnikar, M.; Andrejˇcampa, A.A.; Shaker, R.; Dashti, R. Advanced Edge-Cloud Computing Framework for Automated PMU-Based Fault Localization in Distribution Networks. Appl. Sci. 2021,11, 3100. [CrossRef] 28. Branco, H.M.; Oleskovicz, M.; Coury, D.V.; Delbem, A.C. Multiobjective optimization for power quality monitoring allocation considering voltage sags in distribution systems. Int. J. Electr. Power Energy Syst. 2018,97, 1–10. [CrossRef]