CONSENSORS: A Neural Network Framework
for Sensor Data Analysis
Burkhard Hoppenstedt, R¨udiger Pryss, Klaus Kammerer, and Manfred
Reichert
Institute of Databases and Information Systems, Ulm University, Germany
Abstract. Machine breakdowns in industrial plants cause production
delays and financial damage. In the era of cyber-physical systems, ma-
chines are equipped with a variety of sensors to monitor their status. For
example, changes to sensor values might indicate an abnormal behav-
ior and, in some cases, detected anomalies can be even used to predict
machine breakdowns. This procedure is called predictive maintenance,
which pursues the goal to increase machine productivity by reducing
down times. Thereby, anomalies can be either detected by training data
models based on historic data or by implementing a self-learning ap-
proach. In this work, the use of neural networks for detecting anomalies
is evaluated. In the considered scenarios, anomaly detection is based on
temperature data from a press of a machine manufacturer. Based on
this, a framework was developed for different types of neural networks
as well as a high-order linear regression approach. We use the proposed
neural networks for restoring missing sensor values and to improve over-
all anomaly detection. An evaluation of the used techniques revealed
that the high-order linear regression and an autoencoder constitute best
practices for data recovery. Moreover, deep neural networks, especially
convolutional neural networks, provide the best results with respect to
overall anomaly detection.
Keywords: Anomaly Detection ·Sensor Data Recovery
1 Introduction
In modern industrial plants, a rising number of sensors offers advanced oppor-
tunities for automatically detecting machine faults. In this context, machine-
learning methods have proven to be beneficial [1]. Although first models related
to neural networks were already introduced in the 1940s [2], their usage in an
industrial context only increased significantly during the last years. Thereby,
data storage capabilities constitute an important factor for the development of
neural networks as large training data sets usually enhance the overall outcome.
Furthermore, the developments in the field of deep learning enable a better read-
ability of neural networks. This improvement is achieved by abstracting infor-
mation into hierarchically ordered layers. In this work, we apply neural networks
techniques to an industrial data set, provided by an automotive press company.
2 B. Hoppenstedt et al.
More precisely, the data set contains temperature sensor data from three engines
in a press and the difference between engine temperatures is used to represent
anomalies (cf. Fig. 1, left). These data have been collected over a year of oper-
ation before a machine breakdown occurred. With the help of machine learning
techniques, we tackle two use cases: First, we aim to recognize anomalies. Sec-
ond, we try to recover lost (i.e., missing) sensor data. Regarding the first use
case, the results can be utilized in advanced information systems for the provi-
sion of alerts and condition monitoring. Following the idea that anomalies occur
more frequently before a breakdown, they can be used as input for predictive
maintenance. Concerning the second use case, data recovery is useful to improve
overall data quality and, therefore, enhance data analytics approaches.
Fig. 1. Extract from Measured Sensor Data (left) and Increasing Occurences of Anoma-
lies before a Breakdown (right)
2 Data Set & Evaluation
The data set stems from a machine manufacturer and consists of values from
a press line with six presses. It was recorded over a period of one year, using
a sampling rate of one measurement per minute. At the end of this year, a
breakdown of one press occurred (cf. Fig. 1, right part). To apply methods of
supervised learning, the data points are labeled. Thereby, a single data point was
marked as anomaly if any of the three engine temperature differences was greater
than 5 Kelvin (K). Then, a data section of 120 data points (i.e., each with three
temperature values) was used as the input for the neural networks. Note that
this corresponds to a 2-hour measurement. Furthermore, an input is labeled as
anomaly when it contains at least 105 single data points that are regarded as an
anomaly. Using this approach, we train the neural network to recognize anomaly
patterns. Hereby, we recognize exceeded temperature thresholds as well as the
time series structure. Next, the data set is split up into training and test data.
As the anomalies are underrepresented in the whole data set (cf. Tab. 1), test
data is randomly generated by 1% normal input data and 20% of the anomaly
input data.
The proposed framework hierarchy is shown in Fig 3. It is optimized for sensor
data and includes various types of neural networks. To calculate the precision for
CONSENSORS: A Neural Network Framework for Sensor Data Analysis 3
Table 1. Distribution of the Data Set
ID Data Label Data Set Size Percentage of Parent
1 All Data 524162
2 Normal Data 518095 98.8% of 1
3 Anomaly Data 6067 1.2% of 1
4 Training Data Normal 512915 99.0% of 2
5 Training Data Anomaly 4854 80.0% of 3
6 Test Data Normal 5180 1.0% of 2
7 Test Data Anomaly 1213 20.0% of 3
the use case Sensor Value Restoration, the mean derivation of the expected value
is used as evaluation criterion. For the use case Anomaly Detection, the number
of unrecognized anomalies (false negative) as well as the number of wrongly
discovered anomalies (false positives) are used to calculate the F1-score. These
two evaluation criteria are shown in Fig. 2.
Fig. 2. F1-Scores (left) and Mean Deviation (right)
In Fig 3, the most promising results of the framework are shown. The autoen-
coder (AE) performed well for recovering sensor data, but its variants denoising-
autoencoder (DAE) and compression-autoencoder (CAE) achieve a low F1-score.
In contrast, a fully connected neural network (FCNN) performed better for
anomaly detection than sensor value recovery. Note that the order of the best lin-
ear regression (LR) was four. Finally, convolutional neural networks are promis-
ing for anomaly detection.
3 Summary and Outlook
The presented framework performs well on the shown data set. It restores missing
temperature values with an average deviation of less than 1 Kelvin and recog-
nizes anomaly patterns reliably. Overall, the main advantages of the framework
4 B. Hoppenstedt et al.
are as follows: First, due to the hierarchical class structure, it is possible to
switch between performance and reliability. High-level networks (e.g., the au-
toencoder) need more computation time, but achieve better results. Second, the
framework is designed for sensor data and allows to select a time window to be
evaluated. However, the neural networks need high computation power and, for
supervised learning, a pre-labeled data set is required. In future work, recurrent
neural networks [3] may provide further improvements, as they include previous
decisions into a classification. Other approaches dealing with the application of
neural networks for sensor data exist. For example, [4] discusses an application
of neural networks in a motor system, where vibration signatures are analyzed
with the help of neural network agents. Next, [5] reached F1-scores up to 0.987,
using linear autoencoders to detect anomalies in labeled audio databases. How-
ever, to our best knowledge, the presented framework combined with data of a
press manufacture illustrate new insights.
Interface: ITransformator
LinearRegression
Perceptron
BasicNetwork
NeuralNetwork
Specialized
ConvolutionalNetwork
Autoencoder
Regenerating
Autoencoder
Model Info std(K) std>2k
(Distortion)
AE 0.00 0.95 11.5%
0.01 0.94 10.7%
0.05 0.93 10.5%
(Order)
LR 1 0.88 10.0%
2 0.87 9.8%
3 0.80 8.4%
4 0.79 7.9%
5 0.78 8.6%
FCNN 1.02 12.6%
Model
PER.
DAE
CAE
FCNN
CN(1,0)
CN(1,1)
CN(2,1)
CN(2,2)
F1
0.70
0.70
0.72
0.983
0.988
0.983
0.993
0.972
1 2 3
FCNN = Fully Connected Neural Network, AE = Autoencoder, PER = Perceptron, LR = Linear Regression
DAE = denoising-autoencoder, CAE = compression-autoencoder, CN = Convolutional Neural Network
Fig. 3. Framework (1), Recovery Results (2) and Anomaly Detection Results (3)
Altogether, neural networks are able to adapt to production changes by ad-
justing their weights. Moreover, they are robust to irrelevant noise, as they are
trained to discover only relevant sensor patterns. Therefore, the application of
neural networks is promising in the fields of sensor data recovery and anomaly
detection.
References
1. Zhao, R. et al.: Deep learning and its applications to machine health monitoring: A
survey. arXiv:1612.07640. (2016)
2. Schmidhuber, J.: Deep learning in neural networks: An overview. Neural networks,
61, 85-117 (2015)
3. Gugulothu, N. et al.: Predicting Remaining Useful Life using Time Series Embed-
dings based on Recurrent Neural Networks. arXiv:1709.01073. (2017)
4. Li, B. et al.: Neural-network-based motor rolling bearing fault diagnosis. IEEE trans-
actions on industrial electronics 47, 1060-1069 (2000)
5. Marchi, E. et al.: Deep recurrent neural networkbased autoencoders for acoustic
novelty detection. Computational intelligence and neuroscience (2017)
