Automated IR image Segmentation For Identification Of Overheated Idlers In Belt Conveyor Systems Based On Outlier-Detection Method

Automated IR image Segmentation For Identification Of Overheated Idlers In Belt Conveyor Systems Based On Outlier-Detection Method Mohammad Siamia, Tomasz Barszczb,∗, Jacek Wodeckic, Radosław Zimrozc aAMC Vibro Sp. z o.o., Pilotow 2e, 31-462 Krak´ow, Poland; [email protected] bDepartment of Robotics and Mechatronics, AGH University of Science and Technology, Al. Mickiewicza 30, 30-059 Krak´ow, Poland; [email protected] cFaculty of Geoengineering, Mining and Geology, Wroclaw University of Science and Technology, Na Grobli 15, 50-421 Wroclaw, Poland {jacek.wodecki,radoslaw.zimroz}@pwr.edu.pl Abstract Conveying systems play an essential role in the continuous horizontal transportation of raw materials in mining sites. Regular inspections of conveyor system structures and their components, especially idlers, are essential for proper maintenance. Traditional inspection methods are labor-intensive and hazardous. This paper proposes an automated method to determine overheated idlers. Input data is infrared (IR) and RGB videos of a conveyor system from a mobile robot. Such videos contain disturbances (e.g. sun reflections) which create difficulties to the segmentation of overheated idlers. The method is based on the outliers detection technique. Values of outliers in 8-bit grayscale histogram of extracted frames were considered as optimal threshold values. For the performance validation the shapes and locations of segmented hot spots were compared with RGB frames from the same scene. Furthermore, the segmentation results of four different global thresholding methods have been compared to the proposed method. Keywords: Overheated idlers detection; Maintenance; Inspection robots; IR image; Outlier detection ∗Corresponding author, [email protected] Preprint submitted to Measurement March 6, 2022 This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4054247 Preprint not peer reviewed Automated IR image Segmentation For Identification Of Overheated Idlers In Belt Conveyor Systems Based On Outlier-Detection Method Abstract Conveying systems play an essential role in the continuous horizontal transportation of raw materials in mining sites. Regular inspections of conveyor system structures and their components, especially idlers, are essential for proper maintenance. Traditional inspection methods are labor-intensive and hazardous. This paper proposes an automated method to determine overheated idlers. Input data is infrared (IR) and RGB videos of a conveyor system from a mobile robot. Such videos contain disturbances (e.g. sun reflections) which create difficulties to the segmentation of overheated idlers. The method is based on the outliers detection technique. Values of outliers in 8-bit grayscale histogram of extracted frames were considered as optimal threshold values. For the performance validation the shapes and locations of segmented hot spots were compared with RGB frames from the same scene. Furthermore, the segmentation results of four different global thresholding methods have been compared to the proposed method. Keywords: Overheated idlers detection; Maintenance; Inspection robots; IR image; Outlier detection 1. Introduction Conveyors have been developed and used as the most common system for conveying all forms of material in the mining industry. For decades conveyors have been used for transporting raw materials due to their efficiency and relatively straightforward design. Despite the conveyor advantages, there are still significant challenges for conducting regular inspections to guarantee their operation under harsh environmental conditions in mines [1–5].5 Idlers are important parts of the conveyors that support the belt to carry the material along its full length [6, 7]. Idlers can be damaged by friction, tear, wear, jamming, or seizure. Faulty idlers can become overheated and cause belt damage, thus the temperature, noise emissions, and vibrations of the idler should be constantly monitored through regular inspections. Idlers are located along the conveyor and the typical length of conveyors in the mining tunnel could reach several kilometers [8]. Human inspection of idlers by walking along the belt is time-consuming, costly,10 and hazardous as even a small conveyor of 150 meters consists of nearly 450 carrying rollers and 50 return rollers that should be inspected individually [9]. Monitoring the surface temperature of idlers is a key for finding faulty idlers because the abnormal temperature rise is an important characterization of idlers failure on conveyors. Another critical issue in this particular application is the size of the conveyor. As mentioned, it may be 100m or even 1000m with a large number of idlers to check on a regular basis. The idea of this paper is to provide a quick15 and simple method to collect (inspection robot, see [10–13]) and process (the core of this paper) IR images to identify serious problems with overheated idlers. Several types of faults and conditions in rotating machinery such as coupling looseness, rotor imbalance, misalignment, rolling element bearing damage, and lubricant inadequacy can be detected in the IR image [2, 10, 14, 15]. Although the single infrared image obtained by the infrared camera is widely used for fault detection in rotating ma-20 chines, usually the infrared image has fewer details, low contrast, and lower resolution compared to RGB images, which makes them not suitable for accurate fault detection [16]. Therefore in this paper, we propose to use both RGB and IR images that are captured from the conveyor. Image segmentation is a fundamental step in IR image analysis and it remains a challenge under adverse environmental conditions. Adverse environmental conditions in our case can be defined as the existence of reflective25 objects that can reflect sunlight or the appearance of other hot elements alongside the conveyor[17]. For addressing the mentioned issues and increasing the accuracy of the image segmentation process in this paper we proposed two solutions. Preprint submitted to Measurement March 6, 2022 This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4054247 Preprint not peer reviewed The first solution consists of defining a region of interest (ROI) on extracted key-frames from IR video to minimize the number of hot objects that can be wrongly detected as overheated idlers. Through our examination, we found out30 that even after the definition of the ROI, a considerable number of potential artifacts were still existing on extraxted key farmes. Therefore, as a second solution, we proposed an image segmentation method that worked based on the interquartile range (IQR) outliers detection technique. The proposed methods can accurately detect the overheated idlers even in the presence of potential artifacts. Finally, For validation of segmentation results, locations and shapes of the segmented hot spots are compared with time-synchronized RGB frames that were taken from the same point of35 view by the mobile robot. The paper is organized as follows. First, the problem becomes increasingly explored (as predictive maintenance and inspection robotics are discussed by many authors), thus a comprehensive literature review is provided. It has been divided into several paragraphs as a few perspectives need to be mentioned. Then, an original procedure for overheated idler detection is proposed. Next, we describe the experimental trials and data acquired by the inspection40 robot in the real environment, and finally, the results of the proposed methodology applied to real data are presented and discussed. 2. Literature review For ensuring the safe and most cost-effective operation of industrial machines, regular inspection should be performed by human operators, advanced monitoring systems (SCADAs) [18, 19] or recently also by inspection mobile45 robots [10, 12, 20]. Critical infrastructures are almost always equipped with many sensors and supervisory systems. However, in some situations, as is considered in this paper, condition monitoring systems can be applied on a limited scale only. Drive units (engine, gearbox, etc.) are usually monitored by SCADA[21]. But the rest of the conveyors (for example, a typical conveyor is 1km length), namely moving belt, hundreds of rotating idlers, etc. are difficult to monitor by50 stationary installation [22]. Hence, they are inspected by maintenance staff. Unfortunately, the mining environment is very harsh for that reason, there is a general tendency to minimize the presence of humans. Consequently, the place for the inspection robot is created. 2.1. Mobile robotics for infrastructure inspection Inspection robotics becomes a rapidly growing technology, so the number of papers related to using drones (UAV),55 MAVs [23–27], UGV [11, 28] or legged robots [12, 29, 30] increase every year [23, 25, 31]. Inspection robotics for mining applications is a specific ones, which can be used for exploration, reclamation [25], rescue actions[32], predictive maintenance [11, 28], etc. Regarding the advantages of mobile robots in mining sites, in [20] researchers discussed the reasons that make robotics systems a proper choice for conducting regular inspections in harsh places like deep underground mines.60 Similarly, in [33], the authors discussed the risks that can treat miners in coal mines and proposed solutions based on the application of autonomous mobile robots in inspection missions. In recent years, a considerable number of researches have been published regarding the application of mobile robots for condition monitoring of industrial infrastructures in mines. The work of [10] introduced an unmanned ground vehicle robot for inspection of the belt conveyor system. The proposed robot is controlled by a human operator65 and uses infrared tomography techniques to identify temperature anomalies in conveyor belt systems. In their next paper [11] they developed an automatic robot-based inspection system for 3D modeling and path planning in mines. In [34] researchers proposed a robotics platform for fire identification based on a fusion of data from IR camera, temperature, and smoke sensors. The work of [35] proposed an inspection robotics program for monitoring of the key transmission components temperatures in a conveyor system.70 2.2. IR thermography’s application to industrial infrastructure inspections The IR image can be used for detecting temperature anomalies in industrial infrastructures. More recent work has proposed the application of IR image processing in condition monitoring of industrial infrastructures. In [36] researchers are developed a method for condition monitoring of rotating machineries using thermal image processing. The authors, discussed about the advantages of thermal image based machine health monitoring over the vibration75 2 This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4054247 Preprint not peer reviewed monitoring methods. The proposed method used a convolutional neural network to extract and classify the features for identifying faults in captured IR images. Similar to Jia et al., [37] investigated a fault detection method using infrared IR images for identifying the air compressor pipeline leakage in a mining site. An image processing method for detecting the abnormal heat in underground mine car proposed by [38]. IR and RGB images fusion have been commonly proposed in a wide range of problems related to inspection80 robotics. The image captured by multiple sensors such as thermal, ultraviolet, and RGB cameras can be used for identification and validation of faults in industrial infrastructures [39]. In this context, [40] presented a multisensor information fusion system based on the back propagation neural network (BPNN) for controlling a fire-fighting Inspection robot. In [41], the authors developed a state-of-the-art computer vision methods to identify damaged equipment in power transmission lines. Researchers in [42] proposed a method based on the fusion of the RGB and85 IR images for leakage detection in transmission pipes using a deep learning method. 2.3. Hot spot as an outlier - outlier detection methods Even if inspection robotics becomes increasingly popular, the analysis of data acquired by a robot is still challenging. In this section, we will discuss IR image processing and analysis for hot spot detection with a particular focus on industrial (mining) applications. The general idea is based on extraction areas in IR images with relatively higher90 temperatures in compared to other areas, so a threshold value is required. During the measurements, the inspection robot was able to capture the sequence of IR images without information about the true temperature of the conveyor elements. Due to the automatic scaling of colors to the temperature range, ”hard” thresholding based on the predefined value of temperature was not possible. The novel idea here exploits the concept of outliers. If in a given IR image any hot element will appear in the distribution of pixels, the right tail related95 to ”hot” colors will be heavier than for ”normal temperature” elements. Our target is to detect really hot elements in the conveyor, i.e., significantly higher temperature than other elements in the picture. Thus, our detection strategy was focused on outlier detection in the distribution of pixel color values. Outlier detection is a well-known problem in statistics, data mining, and various applications. Deep reviews one may find in [43–48] 2.4. IR Image segmentation methods - state of the art100 To evaluate the performance of proposed segmentation method, four different auto thresholding methods, namely, Minimum method, Intermodes, Yen, and R´ enyi entropy, were applied to our IR image data set. The Intermodes method is worked based on the assumption that the optimal threshold can be founded in bimodal or multimodal histogramsm as a minimum. It is supposed that the segmented objects are created and a maximum occurs around the most frequent grey level value in an image histogram [49]. Similarly to the internodes, the mini-105 mum method assumes a bimodal histogram. In this method, the histogram is smoothed until only two local maxima remain[49]. The Yen’s method is a multilevel thresholding algorithm that takes into account two factors, the disparity between the thresholded images of the original ones and the number of bits required for the representation of the thresholded image [50]. In the R´ enyi entropy method, the optimal threshold value is defined based on extraction of distributions probabilities of background and object extracted from the grey level distribution of original image [51].110 3. Automatic procedure for overheated idler detections based on IR Video analysis collected by the inspection robot 3.1. General concept Segmentation techniques are useful for the separation of objects of interest from their backgrounds [52]. As shown in Fig.1, to achieve an accurate segmentation of overheated idlers, the following steps were performed: the proposed115 method starts by loading the infrared IR video and extracting the Key-frames. Through the data collection, we received a set of pictures containing many non-informative components related to the conveyor infrastructure. Therefore, in the next step, an ROI is defined on the extracted Key-frames based on the idea proposed by [53]. Afterward, the 24-bit color-coded IR images were converted to 8-bit grayscale image before performing the segmentation process as the proposed thresholding method, requires an 8-bit grayscale image as input. In the next step, the grey scaled histogram120 of the preprocessed IR images were extracted and the outliers values were defined based on the IQR technique. Finally, the optimal threshold value is defined based on the value of the extracted outliers. 3 This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4054247 Preprint not peer reviewed For validation of the segmentation results, the location of identified hotspots in segmented frames are compared to RGB images that are captured from a same point of view. Time synchronization is required for integration of data from RGB an IR camera therefore RGB an IR images are synchronized to be compared based on the same time125 reference. Furthermore to match the image resolution of the RGB to IR images the similar ROI are defined over original captured RGB images. This was necessary, since different camera models with differing image resolutions were used by the inspection robot. Figure 1: The Structure of the proposed method. 3.2. Key-frame extraction from captured IR video Through the conducted examination, the mobile robot continuously record the thermal videos from the conveyor,130 therefore, many frames are almost repeated in a certain time interval. Processing a large number of repeated frames would unnecessarily increase the amount of computational power that is necessary for the segmentation process. Keyframe extraction is a method for expressing the important contents of a captured video file by extracting an important frames [54]. Sampling-Based technique is a method for selecting key frames by uniformly or randomly extracting important frames from the original captured video [55]. This method works based on the idea of choosing135 every Kth frame from the original video. The value of the K parameter can be defined depending on the duration of the video. Usually, from 5% to 15% of the whole video is summarized for extracting key frames. The experimental results show that summarization of 10% video or extraction of every 10th frame gave us enough information to express the main content of the video concisely and reduce the redundant information of the video data [56]. 3.3. Pre-Segmentation of IR image - cropping /selection of ROI for further analysis140 Considering the size of IR images (resolution of 640x480), it would be advantageous if redundant data in nonidlers region can be removed. This reduction has to be performed in a way such that no idlers data is lost while, computational burden is reduced. Therefore, for reducing the amount of unrelated information, an ROI is defined on the original IR frames based on the assumption that IR camera point was remained approximately unchanged during the conducted exam fig. 2.145 4 This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4054247 Preprint not peer reviewed Figure 2: Selection of ROI on the captured IR images 3.4. Segmentation of hot spots based on outliers detection technique The proposed segmentation method, worked based on the fact that in IR images with uniform background, the number of pixels belonging to background or cold areas are much larger than the number of pixels belonging to foreground or overheated objects. Therefore, the foreground representing hot spots will lie at the upper boundary of IR images 8-bit grayscale histogram.150 Outlier detection is a problem of finding patterns in data that are not in the range of normal behavior. In this paper, hot spots in IR images are considered anomalous patterns and treated as outliers. IQR is a technique that helps to find outliers in the data which are continuously distributed. IQR is the difference between the first quartile and the third quartile: IQR =Q3 - Q1. In this method, an image histogram is divided into four parts, first, second, and third quartiles Eq. 1 where N is number of samples.155 First quartile (Q1) =((n+1)/4)th Term Second quartile (Q2) =((n+1)/2)th Term Third quartile (Q3) =(3(n+1)/4)th Term (1) The schematic boxplot is divided data based on four invisible boundaries: two inner fences and two outer fences. The lower inner fence, is computed as Q1 - 1.5 IQR and upper inner fence, is Q3 +1.5 IQR. The, upper and lower part of outer fences can be defines as Q1 - 3 IQR and Q3 +3 IQR, respectively. The mild outliers can be classifies as values that are beyond lower and upper inner fence while extreme outliers are assigned to values that are beyond either lower or upper outer fences [57]. Through examination, we found out that overheated idlers cannot always define as160 mild outliers. Accordingly, for increasing the chance of true detection, the value of extreme outliers are extracted for each frame and considered as an optimal threshold value for segmenting of the overheated idlers Fig 3. 5 This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4054247 Preprint not peer reviewed Figure 3: Comparison of mild and extreme outlier detection results in segmentation of overheated idlers 4. Experiments The experiment has been done in real mining conditions. During the inspection mission, the robotic platform controlled by the operator has captured various types of data including RGB images, IR images, sound, lidar data,165 etc., for various diagnostic tasks, see Fig. 4. Here we will discuss only RGB/IR images. The main parameters of cameras are frames per second, 25fps, resolution of 640x480. The cameras observation angle was 40 degrees, they has been installed on the robot at the height of 100cm above the ground floor. Considered belt conveyor has been in operation in an industrial hall with mineral resource storage, see Fig. 5. The parameters of the conveyors are as follows: length - 150m, belt width - 0.8m, idler spacing - 1.45m, idler diameter170 133mm. 6 This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4054247 Preprint not peer reviewed Figure 4: A general picture of the raw materials storage with belt conveyor to transport raw materials. Figure 5: View of the robot during inspection. During the measurements, several data acquisition sessions have been performed. It is worthy to notice that the environmental conditions are time-varying (even if it is in a kind of indoor condition). A critical issue is that during the experiment, the true temperature of the conveyor elements and IR camera automatically adjust the colors to a given 7 This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4054247 Preprint not peer reviewed temperature range that it produces many complicated images. Based on the manual analysis of the acquired data, we175 have selected several interesting situations that could be problematic during automatic image analysis. Below we present some of such ”difficult to analyze” pictures. In general, one may group them into several classes, namely: images without hot idler, images with hot idler, images with sunlight reflection without hot idler, images with sunlight reflection with hot idler, etc, see the examples presented below Fig. 6. (a) Image without hot idler (b) Image with hot idler (c) Image with sunlight reflection and hot idler (d) Image with sunlight reflection and hot idler Figure 6: Four classes of images that were difficult to analyze 5. Application to real data180 In this section, we will present examples of partial results: raw images, detection results for several images mentioned above. The purpose of this section is to present how difficult is the application of IR image analysis for industrial data, especially in the mining industry. In Fig. 7, one can notice that in the raw IR images there are many non-informative (from the hot idler detection perspective) areas related to, for example, windows or sunlight reflection on the belt (marked by arrows). This will185 make image processing difficult and this is the main reason to limit analyzed area to conveyor related only (cropping the image). In Fig. 8 one can see another example of hot spots (marked by frames) not related to idlers. It shows that the application to real industrial data is always difficult due to unpredictable sources of noise/unwanted components. 8 This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4054247 Preprint not peer reviewed [14] R. Carvalho, R. Nascimento, T. D’Angelo, S. Delabrida, A. G. C. Bianchi, R. A. R. Oliveira, H. Azp´ urua, L. G. Uzeda Garcia, A uav-based310 framework for semi-automated thermographic inspection of belt conveyors in the mining industry, Sensors 20 (8) (2020). [15] W. Yang, X. Zhang, H. Ma, An inspection robot using infrared thermography for belt conveyor, 2016 13th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI) (2016) 400–404. [16] W. Hartmann, S. Tilch, H. Eisenbeiss, K. Schindler, Determination of the uav position by automatic processing of thermal images, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 39 (2012) 111–116.315 [17] S. Henke, D. Karst¨ adt, K.-P. M¨ ollmann, F. Pinno, M. Vollmer, Identification and suppression of thermal reflections in infrared thermal imaging, Inframation Proc 5 (2004) 287–98. [18] A. Zaher, S. McArthur, D. Infield, Y. Patel, Online wind turbine fault detection through automated scada data analysis, Wind Energy 12 (6) (2009) 574–593. [19] A. Grzesiek, R. Zimroz, P. ´ Sliwi´ nski, N. Gomolla, A. Wyłoma´ nska, Long term belt conveyor gearbox temperature data analysis – statistical320 tests for anomaly detection, Measurement: Journal of the International Measurement Confederation 165 (2020). [20] R. Zimroz, M. Hutter, M. Mistry, P. Stefaniak, K. Walas, J. Wodecki, Why should inspection robots be used in deep underground mines?, in: E. Widzyk-Capehart, A. Hekmat, R. Singhal (Eds.), Proceedings of the 27th International Symposium on Mine Planning and Equipment Selection - MPES 2018, Springer International Publishing, Cham, 2019, pp. 497–507. [21] An automatic procedure for multidimensional temperature signal analysis of a scada system with application to belt conveyor components,325 Procedia Earth and Planetary Science 15 (2015) 781–790, world Multidisciplinary Earth Sciences Symposium, WMESS 2015. [22] Research on the fault analysis method of belt conveyor idlers based on sound and thermal infrared image features, Measurement 186 (2021) 110177. [23] J. Shahmoradi, E. Talebi, P. Roghanchi, M. Hassanalian, A comprehensive review of applications of drone technology in the mining industry, Drones 4 (3) (2020) 1–25, cited By 15.330 [24] R. Carvalho, R. Nascimento, T. D’angelo, S. Delabrida, A. Bianchi, R. Oliveira, H. Azp´ urua, L. Garcia, A uav-based framework for semiautomated thermographic inspection of belt conveyors in the mining industry, Sensors (Switzerland) 20 (8) (2020). [25] S. Park, Y. Choi, Applications of unmanned aerial vehicles in mining from exploration to reclamation: A review, Minerals 10 (8) (2020) 1–32. [26] C. Kanellakis, S. Mansouri, G. Georgoulas, G. Nikolakopoulos, Towards autonomous surveying of underground mine using mavs, Mecha-335 nisms and Machine Science 67 (2019) 173–180. [27] C. Kanellakis, E. Fresk, S. Mansouri, D. Kominiak, G. Nikolakopoulos, Towards visual inspection of wind turbines: A case of visual data acquisition using autonomous aerial robots, IEEE Access 8 (2020) 181650–181661. [28] H. Shiri, J. Wodecki, B. Zietek, R. Zimroz, Inspection robotic ugv platform and the procedure for an acoustic signal-based fault detection in belt conveyor idler, Energies 14 (22) (2021).340 [29] C. Gehring, P. Fankhauser, L. Isler, R. Diethelm, S. Bachmann, M. Potz, L. Gerstenberg, M. Hutter, Anymal in the field: Solving industrial inspection of an offshore hvdc platform with a quadrupedal robot, Vol. 16, 2021, pp. 247–260. [30] H. Kolvenbach, D. Wisth, R. Buchanan, G. Valsecchi, R. Grandia, M. Fallon, M. Hutter, Towards autonomous inspection of concrete deterioration in sewers with legged robots, Journal of Field Robotics 37 (8) (2020) 1314–1327. [31] H. Staab, E. Botelho, D. Lasko, H. Shah, W. Eakins, U. Richter, A robotic vehicle system for conveyor inspection in mining, 2019, pp.345 352–357. [32] P. Zimroz, P. Trybała, A. Wr´ oblewski, M. G´ oralczyk, J. Szrek, A. W´ ojcik, R. Zimroz, Application of uav in search and rescue actions in underground mine—a specific sound detection in noisy acoustic signal, Energies 14 (13), cited By 0 (2021). [33] F. G¨ unther, H. Mischo, R. L¨ osch, S. Grehl, F. Gueth, Increased safety in deep mining with iot and autonomous robots, Mining Goes Digital (2019).350 [34] K. Miao, J. Ma, Z. Li, Y. Zhao, W. Zhu, Research on multi feature fusion perception technology of mine fire based on inspection robot, in: Journal of Physics: Conference Series, Vol. 1955, IOP Publishing, 2021, p. 012064. [35] W. Yang, X. Zhang, H. Ma, An inspection robot using infrared thermography for belt conveyor, in: 2016 13th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), 2016, pp. 400–404. [36] Z. Jia, Z. Liu, C.-M. Vong, M. Pecht, A rotating machinery fault diagnosis method based on feature learning of thermal images, IEEE Access355 7 (2019) 12348–12359. [37] K. Tong, Z. Wang, L. Si, C. Tan, P. Li, A novel pipeline leak recognition method of mine air compressor based on infrared thermal image using ifa and svm, Applied Sciences 10 (17) (2020) 5991. [38] J. Du, K. Chen, Q. Liu, J. Wang, Application of infrared thermal imaging technology in fault diagnosis of mine car wheels, in: 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), 2019, pp. 1288–1291.360 [39] Y. Li, F. Yu, Q. Cai, K. Yuan, R. Wan, X. Li, M. Qian, P. Liu, J. Guo, J. Yu, et al., Image fusion of fault detection in power system based on deep learning, Cluster Computing 22 (4) (2019) 9435–9443. [40] [41] B. Jalil, G. R. Leone, M. Martinelli, D. Moroni, M. A. Pascali, A. Berton, Fault detection in power equipment via an unmanned aerial system using multi modal data, Sensors 19 (13) (2019).365 [42] A. Li, D. Ye, E. Lyu, S. Song, M. Q.-H. Meng, C. W. de Silva, Rgb-thermal fusion network for leakage detection of crude oil transmission pipes, in: 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2019, pp. 883–888. [43] V. Hodge, J. Austin, A survey of outlier detection methodologies, Artificial Intelligence Review 22 (2) (2004) 85–126. [44] G. Barbato, E. Barini, G. Genta, R. Levi, Features and performance of some outlier detection methods, Journal of Applied Statistics 38 (10) (2011) 2133–2149.370 [45] H. Wang, M. Bah, M. Hammad, Progress in outlier detection techniques: A survey, IEEE Access 7 (2019) 107964–108000. [46] M. Ernst, G. Haesbroeck, Comparison of local outlier detection techniques in spatial multivariate data, Data Mining and Knowledge Discovery 31 (2) (2017) 371–399. [47] A. Boukerche, L. Zheng, O. Alfandi, Outlier detection: Methods, models, and classification, ACM Computing Surveys 53 (3) (2020). 15 This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4054247 Preprint not peer reviewed [48] A. Smiti, A critical overview of outlier detection methods, Computer Science Review 38 (2020).375 [49] J. M. S. Prewitt, M. L. Mendelsohn, The analysis of cell images*, Annals of the New York Academy of Sciences 128 (3) 1035–1053. [50] J.-C. Yen, F.-J. Chang, S. Chang, A new criterion for automatic multilevel thresholding, IEEE Transactions on Image Processing 4 (3) (1995) 370–378. [51] Threshold selection using renyi’s entropy, Pattern Recognition 30 (1) (1997) 71–84. [52] Chapter 11 - applications, in: Y. Tan (Ed.), Gpu-Based Parallel Implementation of Swarm Intelligence Algorithms, Morgan Kaufmann, 2016,380 pp. 167–177. [53] P. Dabek, J. Szrek, R. Zimroz, J. Wodecki, An automatic procedure for overheated idler detection in belt conveyors using fusion of infrared and rgb images acquired during ugv robot inspection, Energies 15 (2) (2022). [54] C. V. Sheena, N. Narayanan, Key-frame extraction by analysis of histograms of video frames using statistical methods, Procedia Computer Science 70 (2015) 36–40.385 [55] M. Asim, N. Almaadeed, S. Al-M´ aadeed, A. Bouridane, A. Beghdadi, A key frame based video summarization using color features, in: 2018 Colour and Visual Computing Symposium (CVCS), IEEE, 2018, pp. 1–6. [56] S. Jadon, M. Jasim, Video summarization, EasyChair2516-2314 (2019). [57] N. NIST, Sematech e-handbook of statistical methods, 2008, URL http://www. itl. nist. gov/div898/handbook (2019). [58] M. J. Harrap, N. Hempel de Ibarra, H. M. Whitney, S. A. Rands, Reporting of thermography parameters in biology: a systematic review of390 thermal imaging literature, Royal Society open science 5 (12) (2018) 181281. 16 This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4054247 Preprint not peer reviewed