Predictive Maintenance Edge Artificial Intelligence Application Study Using Recurrent Neural Networks for Early Aging Detection in Peristaltic Pumps

IEEE TRANSACTIONS ON RELIABILITY 1 Predictive Maintenance Edge Artificial Intelligence Application Study Using Recurrent Neural Networks for Early Aging Detection in Peristaltic Pumps Juan M. Montes-Sánchez , Graduate Student Member, IEEE,YokoUwate , Senior Member, IEEE, Yoshifumi Nishio , Senior Member, IEEE, Saturnino Vicente-Díaz, and Ángel Jiménez-Fernández , Member, IEEE Abstract—Peristaltic pumps are widely used in many industrial applications, especially in medical devices. Their reliability depends on proper maintenance, which includes the total replacement of tubes regularly due to the aging of the materials. The proper use of predictive maintenance techniques could potentially improve the efficiency of maintenance interventions and prevent failures by having a way to determine when the tube has passed its replacement time. We recorded a dataset using six different sensors (three accelerometers, one gyroscope, one magnetometer, and one microphone) using several cassettes (three new units and three units with expired life span). The recording was done at the highest possible frequency (100–6667 Hz, different for each sensor) and then downsampled several times to obtain frequencies as low as 12 Hz. This dataset is now publicly available. We trained 939 different models, which were the result of combining all different sensors as inputs but the microphone, and four basic architectures of recurrent neural network: One or two layers of either gated recurrent unit or long short-term memory with different number of nodes per layer (from 2 to 64). Among all trained models, we selectedthetenbestperforming networksintermsofbothaccuracy and complexity. All of them reached an F1 score of 0.99 or 1 with holdout cross-validation. Those models were deployed on four different edge AI devices. For all combinations of model and edge AI devices we obtained metrics of memory size (from 0.3% to 160.6% RAM, and from 0.9% to 21.3% flash), inference time (from 0.39 to 1463.91 ms), and average consumption (from 0.15 to 5.30 mA). Nine out of ten models were proven viable for deployment. We concluded that the four models based on magnetometer data were significantly better in terms of consumption and inference time. To the best of our knowledge, the use of magnetometer data is a very uncommonapproachtofailure detection in predictivemaintenance applications, and this is probably the first time it has been used for Received 20 May 2024; revised 8 August 2024 and 26 September 2024; accepted 27 October 2024. This work was supported in part by the Tokushima University between May and July 2023, in part by the Proyecto PREDICAR (ref. ID2023-149777OB-I00) from Agencia Estatal de Investigación, Gobierno deEspañaisasinglesupportingsource,andalsotheonlyoneforthispublication. Associate Editor: Y. Lin. (Corresponding author: Juan M. Montes-Sánchez.) Juan M. Montes-Sánchez, Saturnino Vicente-Díaz, and Ángel Jiménez- Fernández are with the Department of Computer Architecture and Technology, EPS / ETSII, and SCORE lab, Universidad de Sevilla, 41004 Sevilla, Spain (e-mail: [email protected]; [email protected]; [email protected]). Yoko Uwate and Yoshifumi Nishio are with the Nonlinear Circuits Laboratory, Department of Electrical and Electronic Engineering, Tokushima University, Tokushima 770-0855, Japan (e-mail: [email protected]; [email protected]). Digital Object Identifier 10.1109/TR.2024.3488963 peristaltic pump aging detection, so our results are very promising for future applications. Also, since most trained models use little resources, we have proved that our approach is perfectly compatible with running other communication and control algorithms on the same device, which is ideal for easy integration and scalability in industrial systems. Some limitations for real deployment include facing environmental factors (noise) and long-term monitoring, so we also proposed a protocol that should reduce the impact of those factors by taking measurements in a controlled way. Index Terms—Accelerometer, edge artificial intelligence (AI), gated recurrent unit (GRU), industrial Internet of Things (IIoT), long short-term memory (LSTM), magnetometer, peristaltic pumps, predictive maintenance (PdM), recurrent neural network (RNN). NOMENCLATURE AI Artificial intelligence. EC Edge computing. GRU Gated recurrent unit. IDE Integrated development environment. IIoT Industrial Internet of Things. IMU Inertial measurement unit. LST M Long short-term memory. MCU Micro controller unit. ML Machine learning. PdM Predictive maintenance. RAM Random access memory. RNN Recurrent neural network. I. INTRODUCTION EVERY industrial equipment needs periodic maintenance to ensure proper functioning. Although the maintenance process itself has a cost, not doing it correctly reduces the life expectancy of the equipment and may cause catastrophic failures. Reducing maintenance costs while also reducing the chances of malfunction has been the goal of all experts and companies researching in this field. Recent development of technology for automation, measurement, and communication has provided the perfect scenario for the emergence of predictive maintenance (PdM), which aims not only to prevent failures, but also to predict when the failure is more likely to occur so the preventive interventions could be done in a more efficient way [1]. © 2024 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 2IEEE TRANSACTIONS ON RELIABILITY TABLE I RECENT RELATED PDMWORKS A. PdM and Industry 4.0 PdM has also become one of the key aspects of the so-called Industry 4.0 and industrial Internet of Things (IIoT). Many authors have published about PdM systems in recent years, linking this technique with the terms Industry 4.0 or IIoT, and usually using or at least mentioning artificial intelligence (AI) techniques (see Table I). The authors in [2] and [3] conducted a comprehensive analysis of a selection of works whose authors applied machine learning (ML) methods to solve PdM challenges (among other applications in industry) and concluded that, although there are some difficulties in developing and implementing these algorithms, there is a clear trend that shows an increasing number of successful implementations and also a growing interest not only in the academic world but also in industry in recent years. Also, Ahmed et al. [4] conducted an in-depth review of AI methods used in Industry 4.0 and PdM wasmentionedasone of themainapplications, remarking its importance and benefits when applied to fault detection, condition monitoring, or production management. Those benefits include price savings, longer predictability, and increased availability. Abidietal.[5] made an analysis of previous AI approaches for PdM using several different methodologies, identifying the features and challenges of each, and then proposed a novel method also using an AI approach. In conclusion, although it is not the only possible approach, all these cited works acknowledge the convenience of using AI in many of its forms for PdM tasks, although the exact method of deploying it in the field is not always specified. B. Edge AI For handling data in industrial environments, which is one of the key aspects of IIoT, cloud-based solutions are generally used together with edge computing (EC). The use of computingcapable devices located in the same place where data are collected has many advantages compared to architectures that do not use EC [15],[16]. EC is very useful for reducing network traffic in high volume sensor data scenarios by preprocessing data before sending sensor information to cloud computers in IoT systems [17].EC systems can also improve privacy by filtering which data are sent to the cloud [18]. This is critical in the use of audio for classification, as part of the ambient noise could be sensible information. If properly implemented, the reliability and availability of a system increase when EC devices are present [19]. When these devices make use of AI networks that run directly on the device itself, it is called edge AI. Not all previous PdM works point to edge AI as directly related to or useful for their applications, but it is very common among the ones cited previously and many others, since it is a very convenient and power-efficient way of deploying PdM solutions. Edge AI is used in many scenarios involving sensing and classification, not only in industry [20] but also in wearables [21]. It offers a faster and more stable response than cloud-based implementations, but can also complement these [22], and can run on very simple computers, reducing not only economic costs but also power consumption [23]. The con is that more complex AI algorithms need optimization and the This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. MONTES-SÁNCHEZ et al.: PREDICTIVE MAINTENANCE EDGE AI APPLICATION STUDY USING RNNS FOR EARLY AGING DETECTION 3 accuracy of classifications could be worse in some cases due to model compression [24]. Outside of the academic world, edge AI is currently being considered a fast-growing and soon to reach general usage in industry by Gartner among all AI fields [25], and was one of the technologies that stood out according to Deloitte’s predictions for 2020 [26]. Industry has also shown interest in edge AI by supporting publications on the topic. Some notable examples include the Bosch white paper of 2018 [27] or the publication on behalf of the European Association on Smart Systems and Integration of 2021 [28]. C. Peristaltic Pumps Regarding the scenario proposed in this work, which involves peristaltic pumps in medical applications (described in detail later in Section II-C), it is mandatory to introduce some of the concerns raised by the use of these devices in recent years. Peristaltic pumps come in many different configurations and are ideal for medical applications, as these pumps do not contaminate the fluid they transport, but also present some disadvantages related to calibration and degradation, so the application of control techniques is very important [29]. Most of the issues related to peristaltic pumps come from the need to squeeze the tubes to move the liquid, which causes stress and damage to it. This damage can result in the generation of particles coming from the tube itself that contaminate the liquid or even in catastrophic failure [30]. Some authors have identified some of these existing problems and have studied novel ways of designing peristaltic pumps to improve longevity or reliability, such as [31] and [32], among others, focusing on reducing the impact of mechanical parts on the tube. Tube degradation occurs in a different grade depending on the materials used, the diameter of the tube, and the pump running time [33]. That is why some peristaltic pump vendors usually specify an expected tube lifetime, which is only an average estimation under certain circumstances, and since they cannot guarantee the correct operation after that period, they recommend the full substitution of the tube at least before that estimated time passes [34]. The piece containing this tube that needs to be replaced is often called cassette, and it is made so that it can be easily attached or detached from the motor. There is even a recent study of PdM decisions for an industrial thermoforming equipment which had peristaltic pumps as one of its components. Due to a failure of the pumps that caused the machine to stop working, the authors corrected the mean time to failure of the peristaltic pumps, as it was shorter than advised, and adopted a different maintenance strategy for them to prevent this situation [10]. Although they decided not to implement any PdM method for this part, it is a good example of the consequences of failure occurring in peristaltic pumps even when preventive maintenance is performed as recommended. All these issues related to peristaltic pumps and the lack of literature covering PdM in this scenario are the main reason to choose a hydraulic module based on peristaltic pumps from a biomedical device to apply AI-based PdM techniques for early detection of degradation symptoms in the tube. D. Contributions of This Work The main contributions of this work include the following. 1) Using peristaltic pumps coming from real equipment used in a biomedical device for creating a public dataset including labeled data for brand new pumps (tube in good condition) and old pumps (tube lifetime already expired) running in the same conditions. This dataset contains measurements from several sensors: accelerometer, gyroscope, magnetometer, and audio. It will be available for all science community under FAIR data principles [35]. 2) Comparing up to 939 different networks trained with this dataset, resulting from different combinations of five variables: sensor, sampling frequency, type of neural network, number of layers, and number of nodes. 3) Proposing a simple method to select the best network for field usage by rating the networks in terms not only of accuracy but also complexity. 4) Evaluating the ease of these networks to be deployed on four different microcontroller units (MCU) for edge AI computing, each one representing a different family with different characteristics, taking into consideration inference time, memory usage and power consumption. 5) Proposing a protocol for real use of the system developed in this work during in-field operation. E. Structure of This Work The present work is structured as follows. 1) First, we present the methods used and the equipment. 2) Second, we present the equipment used in our PdM scenario and the categories we wanted to classify, as well as the network architecture. 3) Then, we describe the creation of the dataset and the data analysis. 4) After that, we explain the method used to select the best network. 5) Afterward, we evaluate some of the best networks for their use in several edge AI devices. 6) Later, we propose a protocol for real use of our system (in-field operation). 7) Before the conclusions, we identify some limitations and future work for addressing those. 8) Finally, we discuss the conclusions of this work. II. MATERIALS AND METHODS A. Gated Recurrent Neural Networks (RNNs) RNNs are different from a feedforward neural network, where the information between its layers flows in one direction, in the way that the output of some nodes can also flow backward into the layer or layers after them. This feature, also called memory, is ideal to process time-dependent data, but as more data flows through the network the information of the first operations becomes almost irrelevant [36]. Gated RNNs try to solve this issue by introducing some memory cells that can store information so that it is not lost. If this information is stored or not, it is managed through a set This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 4IEEE TRANSACTIONS ON RELIABILITY of gates, which can be opened or closed by the cells. Training sets how cells operate these gates according to the relevance of the data [37]. Among the possible implementations of gated RNNs, long short-term memory (LSTM) and gated recurrent unit (GRU) are widely used. 1) LSTM: LSTM units [37] have three different gates, called the input gate, the forget gate, and the output gate. The input and forget gates can add or delete stored information, and the output gate controls if this information is sent to the next step. 2) GRU: A more recent approach of gated RNN, which is a simpler version of an LSTM cell, is the GRU [38]. Its main difference from LSTM is the absence of an output gate, so all information stored in the memory passes to the next layers in every step. GRU and LSTM are very similar in terms of effectiveness, but GRU, being less complex, is usually easier to run, since it requires less computing resources [39]. 3) Limitations of RNNs: Gated RNN mechanisms allow long data sequences to be taken into account, but they are still not sufficient to learn long-term patterns in sequences of 100 time steps or more [40]. To solve this problem, other implementations are being developed, such as the WaveNet architecture [41]. Another limitation of RNN is that, due to its sequential nature, training RNN networks is costly in terms of resources and time. There are novel approaches that try to improve training time without losing performance, such as the work in [42].Onthe contrary, this improvement in training time with superior performance results in a more complex trained model than choosing LSTM or GRU, which would be more difficult to run into edge AI devices. B. Proposed Architectures Since the final objective of this work is to embed an RNN- based classification algorithm in a small device, the architecture of the network should be as simple as possible, while trying to maintain enough processing layers for the classification to be successful. Other authors faced this same problem before, also while having three-axis measurement windows as input [43]. By adding additional layers to a basic architecture with an RNN layer, they found that a second RNN layer can improve classification results, but other additional layers had very little impact on accuracy. This study was focused on human activity rather than motor monitoring, so we think it is also a good opportunity to see if the good results of the RNN architectures shown by the authors are suitable for a different application. Following the results of this work, we decided to test the following architectures. 1) Input layer. 2) Batch normalization layer. 3) One or two sets of gated RNN layer (GRU or LSTM) followed by a dropout layer [44]. 4) Dense layer. 5) Softmax layer. 6) Output (one of our three classes). By changing the type of RNN (GRU or LSTM) and the number of these layers (one or two), we ended with a total of Fig. 1. Architecture of the networks we tested. Nodes1and Nodes2are the number of nodes for these layers, which ranges from 2 to 64. four basic architectures to test. In all these four architectures the number of nodes was also increased, starting with two nodes per layer and ending with 64 nodes per layer, in an exponential increase. Adding additional nodes and layers makes the algorithm more complex as the total parameters increase. Taking into account the variation of the nodes, this produces a total of 24 different RNN architectures for evaluation. See Fig. 1. C. Experimental Setup We used a hydraulic block from a commercial biomedical equipment consisting of three Thomas SR10/30 dc Standard peristalticpumps[34],fourvalves,andtubingmountedonasteel frame(seeFig.2). The frame was fixed to a stable rigid leveled surface. The tubes were filled with distilled water. All pumps and valves were connected to a control board, but we modified the firmware of this board so that only one pump was turned ON at the same time in our testing scenario, selected at random. The pumps always ran at the same constant speed (full speed of the motor when powered up with a 12 V power supply). This limitation comes from the system itself, which was not designed to be controllable in speed. We also used several cassettes: three completely new and the other three used ones, which have already passed their recommended life span. All of these cassettes were changed in the running pump between data samples as needed to sample all different categories. This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. MONTES-SÁNCHEZ et al.: PREDICTIVE MAINTENANCE EDGE AI APPLICATION STUDY USING RNNS FOR EARLY AGING DETECTION 5 Fig. 2. Hydraulic block from the biomedical equipment during dataset recording. TABLE II AVAILABLE SENSORS Using high-quality double-sided tape, we attached to the upper side of the frame an ST SensorTile.box wireless multisensor development kit [45]. This device was programmed with the custom firmware FP-SNS-DATALOG1 provided by the vendor [46], which allows high-speed data sampling from all sensors. The device includes the following sensors, which are all available for data recording (see Table II). The most used sensors for motion classification among the ones this device has are the inertial sensors (accelerometers and gyroscope) and the audio sensor (microphone). Accelerometers and gyroscope data were all included in the network training phase, but not audio data. This is because audio classification is not part of this work, as the nature of the data is very different in shape and format and would require the use of different methods. Besides that, we included the sensor data in the dataset recordings for future studies, as many works involving motors use this approach [13]. It is uncommon to use magnetometer sensor data for motion classification, although there are some recent examples, such as the Koch et al. [47]’s work or the Belalcazar-Bolaños’s [48] work, who designed a method for detecting magnetic disturbances in IMU sensors using magnetometer data as input. Other previous examples demonstrated that magnetic disturbances can be used as an input for passing vehicle classification [49], and also that frequency analysis of magnetometer data close to a running motor showed that it could be used for fault detection [50]. Thislastexamplefrom2015istheclosestonetoourscenario,but to our knowledge no specific detection method was published for doing so after this work, so magnetometer data have not been compared yet to other well-known methods for motor fault detection in equal terms. After reviewing all these works, we concluded that it could be possibleto detect patterns inthismagneticfield disturbances that the running motor of the pump could produce by using RNN- based classification. In our scenario, the shape of the data is the same as that of the accelerometer and gyroscope sensors. For all thesereasons,wealsoincludedthereadingsofthemagnetometer sensor in our network training. III. DATA GATHERING AND PROCESSING A. Classes We considered the following classes. 1) STOP: No pumps running. No movement. 2) NEW: One pump running at default operation speed with a new cassette in good condition. 3) OLD: One pump running at default operation speed with an old cassette. B. Recording Process Taking advantage of the capabilities of the software provided by the vendor, which includes full wireless control of the device and manual data labeling during the recording process, the recording of raw sensor data for three different situations (labeled STOP, OLD, and NEW) was performed as described in the following pseudocode. 1) With one initial cassette installed in the randomly selected pump (for example, a new one), and the tubing filled with liquid, the recording starts. 2) Since the pump has yet to be turned on, after a few seconds the STOP category label can be activated, and STOP-labeled data are acquired for all sensors. 3) After the desired sequence time, the STOP category label is disabled, ending a data sequence. 4) The selected pump is activated. 5) Afterafewsecondsthe label corresponding to the NEW or OLD category, depending on which cassette is installed, can be activated for recording labeled data for all sensors. 6) After the desired sequence time, the label is disabled, ending another sequence. 7) Then, we can stop the pump, go back to the first step, and repeatthesequence all the times we desire, with orwithout stopping the data recording, since we can then delete all unlabeled parts. 8) When we finish, all we must do is stop the recording and the raw data will be stored on the SD card of the device. 9) Data files are transferred to the PC for further processing. C. Dataset Creation We coded a set of Python scripts for processing the data. 1) Discard Unlabeled Data: Using vendor software libraries andaddingmorecodefromourown,analgorithmwasdeveloped to convert raw sensor data files to clean data in comma separated values text format. Since the raw data already contain valid labels, the first step was to automatically discard all unlabeled This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 6IEEE TRANSACTIONS ON RELIABILITY data. This was done carefully so that the different data sections remained independent of each other. 2) Data Normalization: We did not apply any data normalization technique during the preprocessing. This is because we prefer to keep the data as close as possible to the raw lectures of the sensor. There are some advantages of doing so: First, since the sensors provide the lectures in its raw form, by feeding the trained network with it directly, we avoid adding an extra computing step that can cause a delay. This is especially critical in edge AI devices with low computing capabilities. Second, by publishing the nonnormalized data, we allow other researchers to apply their own normalization techniques in the future should they wish to do so. 3) Frequency Downsampling: The data were recorded at the highest possible frequency, only limited by the characteristics of the sensor. This is not the only way the sensor data can be acquired, since all of them can be adjusted to sample at lower frequencies.Samplingatalowerfrequencymeansthatthesensor waits longer between taking lectures. Instead of repeating the wholerecordingprocess,whichisnotideal,sinceitisimpossible to replicate the exact behavior of the pumps two times in a row, we made postprocessed downsampled datasets. We halved the frequency several times, ranging from the maximum frequency to a minimum frequency of 10 Hz. Although it is possible, going below 10 Hz was not considered necessary, since it is already a very slow sampling rate. 4) Data Extension by Overlapping Windows: For training, data need to be sliced into time windows (a group of consecutive time samples of a determined length). We decided to split the data into fixed-size windows. This is usually done in one of two ways: by overlapping or not overlapping the windows. We have chosen to overlap windows by the middle for two reasons: First, doing so results in having more data. Second, by doing it that way, there is no way that key characteristics should only be considered at the beginning or end of the sample, which could result in having that characteristic always sliced between windows. Anyway, according to recent studies in three-axis accelerometer datasets [51], it is not worse to have overlapping windows compared to not doing so, in fact, it could be better. If the training is done using subject-independent cross-validation, which we did, there should be no difference. We first tested one of the datasets to fix the size of the time windows. After testing the 1, 2, and 0.5 s window sizes, we concluded that the results were very similar. We decided to slice the data in 0.5 s window sizes since it is the smallest of the ones we tested, keeping the number of lectures in each sample lower also at higher frequencies. Doing so is important for LSTM and GRU cells to function well. A graphic example of three overlapping windows can be seen in Fig. 3. 5) Train, Validation, and Test Data Separation: We collected and processed two separate datasets. The first dataset is the largest (more than 1000 windows per sensor) and contains classified data from three different peristaltic pump cassettes (a new one and two old ones). It was named the train dataset, as it was later used for network training and validation by splitting it into two parts (85% of the data for training and 15% for validation). The second dataset is smaller (more than 600 windows per Fig. 3. Example of three 0.5 s length overlapping windows, each one represented in a different color and numbered 1–3, over a simulated signal with random values, for illustration purposes. By doing this technique, three different 0.5 s samples are obtained from 1 s of data. TABLE III AUGMENTED DATASET DISTRIBUTION sensor) and contains classified data from two peristaltic pump cassettes (a new one and two old ones), which were different from the ones used in the train dataset. This second dataset was namedthe testdataset,as itwastheoneused toobtainthe metrics of the networks resulting from the training. Both datasets are considerably balanced in terms of the number of samples per tag, and the split between train and validation was made also maintaining this balance. Table III shows the amount of data we used for training, validation, and testing in the augmented dataset (after applying data extension techniques). 6) Final Dataset Format and Publication: We made all the recorded data available for public distribution under the principles of FAIR data [35]. To do so, we exported the recorded data for all sensors to separate files for each sensor and sample, including the audio data from the microphone. The data we made available are not augmented: It is kept as it was before the data extension techniques applied in Sections III-C3 and II- I-C4 in case other authors want to test different methods. All data were uploaded to the Universidad de Sevilla’s public data repository [52]. IV. NETWORK TRAINING For the model training, we used a Python environment on a personal computer (PC) equipped with a NVIDIA GeForce RTX 3060 graphics processing unit where we used Keras libraries to obtain the best model for each proposed dataset and architecture. This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. MONTES-SÁNCHEZ et al.: PREDICTIVE MAINTENANCE EDGE AI APPLICATION STUDY USING RNNS FOR EARLY AGING DETECTION 7 TABLE IV HYPERPARAMETERS COMPARED TO THE RELATED WORK [13] The versions we used are as follows: Python 3.10, TensorFlow 2.9.1, and Keras 2.9.0. Hyperparameter configuration for training is drawn in Table IV. From all related works previously cited in Table Ithe one that presented its configuration for a similar architecture, also based in LSTM and GRU, is [13]. Therefore, we added those values next to ours for comparison. We repeated the training process a minimum of five times for each model, randomly dividing the data between train and validationeachtime(althoughkeepingitbalancedintermsofthe amount of data for each class). Then, to test the obtained model, we used as input the corresponding test dataset, which was never used during training, also known as holdout. We calculated the following metrics [53] for each model we obtained using the holdout data: accuracy = c TPc+TNc TPc+TNc+FPc+FNc ,c∈classes (1) sensitivity = c TPc TPc+FNc ,c∈classes (2) specificity = c TNc TNc+FPc ,c∈classes (3) precision = c TPc TPc+FPc ,c∈classes (4) F1score =2×precision ×sensitivity precision +sensitivity (5) where classes are the three different classes we defined, TP and TN are true positive and true negative, respectively, and are related to the number of samples correctly classified, FP and FN are false positive and false negative, respectively, and are related to the number of samples incorrectly classified. We obtained F1 score for each class, but the metric we used to determine the performance of the model was the F1 weighted average. This is a mean of all three F1 scores of our model (one for each class), but taking into account how many occurrences of the class are in the test dataset. The number of occurrences determines the weight of each F1 score of each class in the mean calculation. We also calculated F1 macroaverage, which is an unweighted mean of all three F1 scores. In addition, we generated the progression of model accuracy per epoch graph and the confusion matrix for each trained model. In Fig. 4, we can see examples of those for a randomly selected model (accelerometer LSM6DSOX samples at 1666 Hz for an architecture with two layers of GRU with 32 and 16 nodes, respectively). Fig. 4(a) shows a typical training where the accuracyisverybadinthefirstepochsbutimprovesveryquickly, and then it reaches its maximum with very little progress in the last 800 epochs, where it remains stable at almost 1.0. Fig. 4(b) shows how well that same model classifies the holdout data (which were not used during training). We used these graphical data for review and control after each training session. In Fig. 5is represented the mean F1 weighted average of a selection among all trained models, grouped by the same characteristics. This is the metric we used to determine which model performed the best classification. The value of this metric is shown for each sensor, frequency, architecture, and number of nodes. We can observe that, for higher frequencies, it is very difficult for the model to converge during training due to the larger amount of data in each sample. Repetition of the process several times eventually leads to much better results. We wanted to represent this inconsistency but also the highest value possible, which is indeed obtainable. For that reason, we also show in the same graph the maximum and minimum F1 weighted average obtained for a trained model, represented by the top and bottom of the black vertical lines inside each bar. V. EDGE AI DEVICE Aside from the metrics obtained for each model with (1)–(5), we considered other relevant factors before choosing which model is the best to be deployed in a final edge AI device-based system. A. Computational Complexity Since an edge AI device has limited computing resources, we need a way to estimate which models are easier to run. We considered two main factors as follows. 1) Number of Parameters: Adding additional nodes and layers makes the algorithm more complex as the total parameters increase. More parameters means the algorithm needs more random access memory (RAM) size and computing cycles for each classification. This number of parameters only depends on the architecture and number of nodes, so it is the same for all identical configurations, regardless of the type of sensor or frequency. See Fig. 6for a graphical comparison. 2) Sampling Frequency: It isdirectlyrelated totheamount of data that both the sensor and the microcontroller need to process each second. Therefore, a lower sampling frequency means that less resources will be needed to obtain sensor data. B. Selecting Best Suitable Models for Edge AI Evaluation We have designed the following method for selecting the models that can be considered for edge AI evaluation, described by the following pseudocode. This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 8IEEE TRANSACTIONS ON RELIABILITY Fig. 4. Example of accuracy progression during training (a) and confusion matrix (b) for accelerometer LSM6DSOX samples at 1666 Hz for an architecture with two layers of GRU with 32 and 16 nodes, respectively. TABLE V BEST MODELS SELECTED WITH PROPOSED METHOD 1) First, we select a kind of sensor among the three considered: accelerometer, gyroscope, or magnetometer. We treated each type individually. 2) F1 weighted average is selected as the primary metric for determining the best classification results. 3) Since each model has been trained five or more different times, only the model with the highest F1 weighted average is considered for each different network. 4) Then, we set a threshold for the minimum value required for F1. In this case, it was set to 0.99. All models with a F1 weighted average under 0.99 were discarded. In total, 33 magnetometer data models and 30 accelerometer data models remained. All gyroscope-based networks are discarded in this step because none of them reaches such a high F1 score. 5) We sorted the results starting from the lower sampling frequency. 6) For each sampling frequency, we selected only the less complex network. Furthermore, since there were several models that achieved perfect classification during tests, we decided to repeat the selection method but with the threshold of F1 weighted average set to 1 (the maximum possible value) instead of 0.99. By doing so, only 11 magnetometer data models and ten accelerometer data models remained. In fact, for accelerometer data, only the LSM6DSOX sensor at very high frequencies of 3333 and 6667 Hz achieved these accuracy levels. However, the magnetometer data showed very good consistency among all frequencies, as seen in Fig. 5, so the new criteriaonly changed theselectedmodelsinterms of complexity, maintaining the same frequencies as with threshold 0.99. A comparison of selected models with thresholds 0.99 and 1 can be seen in Table V. These models were the ones we put to test in real devices for the final comparison. The devices selected for evaluation are each from a different series of STM32 32-bit Arm Cortex MCUs [54]. We have chosen two different MCUs among the high performance range, each one based on a different Cortex-M core, and another two among the ultralow power range, also based on different Cortex-M core. Those MCUs can be purchased already mounted on evaluation boards, ready for use. The exact four selected boards, with their corresponding MCUs and characteristics, are resumed in Table VI. This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. MONTES-SÁNCHEZ et al.: PREDICTIVE MAINTENANCE EDGE AI APPLICATION STUDY USING RNNS FOR EARLY AGING DETECTION 9 Fig. 5. Selection of mean F1 weighted accuracy for different models. The black bars represent maximum and minimum values obtained. C. Edge AI Devices Evaluation 1) RAM Memory Size and Inference Time: We used the integrated development environment (IDE) STM32CubeIDE from the vendor ST [55], along with the four selected evaluation boards from this same vendor, for a comparison of memory, inference time and power consumption. This IDE includes tools for deploying previously trained neural network models on the boards and then running them in real time, obtaining the processing time required to complete one classification (inference time)and the RAM memory sizeof the model plus theadditional libraries required. This tool has an option for optimizing the This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.