Fusing convolutional generative adversarial encoders for 3D printer fault detection with only normal condition signals

Fusing convolutional generative adversarial encoders for 3D printer fault detection with only normal condition signals Chuan Li a,1 , Diego Cabrera a,b, ⇑ ,1 , Fernando Sancho c , René-Vinicio Sánchez b , Mariela Cerrada b , Jianyu Long d , José Valente de Oliveira e a National Research Base of Intelligent Manufacturing Service, Chongqing Technology and Business University, Chongqing 400067, China b GIDTEC, Universidad Politécnica Salesiana, Ecuador c Dpt. of Computer Science and Artificial Intelligence, Universidad de Sevilla, Spain d School of Mechanical and Automotive Engineering, South China University of Technology, Guangzhou, China e Universidade do Algarve, Portugal Keywords: Fault detection 3D printers Condition-based maintenance Convolutional neural networks Adversarial learning abstract Collecting data from mechanical systems in abnormal conditions is expensive and time consuming. Consequently, fault detection approaches based on classical supervised learning working with both normal and abnormal data are not applicable in some conditionbased maintenance tasks. To address this problem, this paper proposes Fusing Convolutional Generative Adversarial Encoders (fCGAE) method to create fault detection models from only normal data. Firstly, to obtain an adequate deep feature space, encoder models based on 1D convolutional neural networks are created. Then, these encoders are optimized in an unsupervised way through Bidirectional Generative Adversarial Networks. Finally, the multi-channel features collected from the system are merged with One-Class Support Vector Machine. fCGAE is applied to fault detection in 3D printers, where experimental results in two fault detection cases show excellent generalization capabilities and better performance compared to peer methods. 1. Introduction 3D printing has been attracting much attention as an innovation technology in the industry [1], as well as in medical [2], food [3], and other areas [4]. This is due to the accuracy level achieved by this technology [5], which improves the quality of the obtained products. Of course, the quality that a 3D printer offers depends mainly on the transmission elements that constitute it [6]. Despite the fact that the quality of the components of a 3D printer is the best or not, these elements suffer from wear and tear due to continuous use in the printing process, as in any other machinery [7]. Additionally, the union of the components can be compromised by the loosening of the fastening elements such as nuts, bolts, and join bearings. Those wear and/or loosening cause unintended movements in the printer head along with unwanted vibrations of the machine [8], resulting into loss of print quality, an accelerated deterioration of other components of the printer [9], and possible risk of damage ⇑ Corresponding author at: National Research Base of Intelligent Manufacturing Service, Chongqing Technology and Business University, Chongqing 400067, China. E-mail address: [email protected] (D. Cabrera). 1 The two authors contributed equally to this paper. to the machine or operator safety. Hence, it is critical to estimate the condition of the machine in order to make necessary adjustments or replace defective parts. Several efforts have focused on fault detection and diagnosis for 3D printers. For example, in [10] a method to combine different sources of information was proposed for estimating the printing quality. The estimation of the machine condition itself was not performed, but only informed if an error occurred during the printing process. In [11], an approach based on the filtering of acoustic emission signal and the Kurtogram extraction of a photoelectric signal was considered for the fault diagnosis in additive manufacturing. In that work, features are extracted considering machinery expert knowledge. With the same application, in [12] fault diagnosis was proposed by using Support Vector Machines (SVM) with signals obtained from an attitude sensor. In [13,14] the 3D printer condition is estimated by Echo State Networks (ESN), while [15] introduced a method improving an extreme learning machine through a modified swarm optimizer, and [16] presented the feature reinforcement for improving the 3D printer condition classification with a success rate in the normal condition of 93.6%. In addition, the comparison results reported 66.7%, 67.5%, 90.1%, 32.0% and 21.3% with SAE+Softmax, ESN, SAE+ESN, SAE+SVM, and SVM, respectively, highlighting the difficulty to correctly detect a fault even with powerful pattern recognition algorithms and supervised learning approachs. [17] introduced a method for unsupervisedly optimizing the reservoir of an ESN and applied it to this problem. And [18] used a sparse autoencoder network (SAE) for fault diagnosis of a delta 3D printer. Authors from [19] presented transfer SVM for fault diagnosis of 3D printers in light of the lack of data on fault conditions by transferring learning from other domains with more available data, achieving a classification accuracy of 83.79%. [20] introduced the sensor fusion by error merging of SAE to dynamically assess the condition of a 3D printer. These last two works are the only ones that address the problem of lack of data for training classification/regression models for 3D printers with promising results. More works on fault diagnosis and fault detection have been reported for other machines. Approaches based on stochastic resonance for detection and diagnosis of machinery failures were presented in [21–23]. Techniques for fault diagnosis based on signal analysis were detailed in [24], with emphasis on the extraction of periodic impulses in [25] and analysis in the time–frequency domain in [26]. In addition, a considerable amount of works using artificial intelligence techniques have also been reported. For example, [27,28] compared several approaches based on fuzzy logic for bearing diagnostics, and [29] presented automatic feature extraction by convolutional auto-encoder to deal with feature engineering in helical gears. A more exhaustive review can be found in [30]. All these works do not consider the difficulty of obtaining abnormal data of the machinery to create a classification model through supervised learning, but some other works have also been reported addressing this problem. For example, in [31] an Echo State Network was combined with a Variational Auto-Encoder for the learning of a detection model only from normal condition data of a helical gearbox. [32] presented Generative Adversarial Networks (GAN) as a model able to capture the data manifold in normal condition using its discriminator model. In [33] Categorical Adversarial Auto-Encoder model was presented for data clustering where condition was unknown. [34,35] used One-Class SVM (OCSVM) model for the detection of bearing failures, and [36] introduced an improved SVM for aircraft engine fault detection with imbalanced data. However, it remains a challenging task to detect failures in new domains with only data from normal condition, where expert knowledge is not available to apply feature engineering-based approaches to determine machinery condition, and where the sources of information are diverse. This paper proposes a way to solve this challenge through fusing Convolutional Generative Adversarial Encoders (fCGAE) applied to fault detection in delta 3D printers. Main contributions of this work include: (i) a Convolutional Generative Adversarial Encoder (CGAE) model that integrates 1D Convolutional Neural Networks (1DCNN) with Bidirectional Generative Adversarial Neural Networks (BiGAN), in order to map raw signals to an informative feature space for fault detection; and (ii) a proposal for the fusion of deep features extracted in a non-supervised CGAE from multi-channel signals, and the creation of a discriminant function from them. The results obtained in fault detection in 3D printers show a better performance of fCGAE compared to approaches reported in the literature using classical feature engineering and state of the art Deep Learning. The paper is organized as follows. Section 2presents the details of the proposal. Experiments for the validation of the method are presented in Section 4. The results and comparisons obtained in each fault diagnosis study case are described in Section 5. Finally, Section 6presents some conclusions and future work. 2. Methodology 2.1. Signal augmentation The operating condition of a 3D printer can be inferred through measurements of sensors collecting information of different variables. In this research, both 3-axis velocity sensors and 3-axis angle sensors are considered: sðtÞ¼fs i ðtÞg 6 i¼1 ð1Þ where s 1 ;s 2 ;s 3 ;s 4 ;s 5 ;s 6 denote, respectively, the x;yand zvelocity and x;yand z-related angle signals. By using the window slicing technique for data augmentation in time-series datasets [37], and choosing appropriate values for the length of the slices ( s l ) and the time-steps before the next slicing ( s s ), we can extract Kshort signals from every original one, obtaining a larger set of smaller signals (that can overlap): s#fs ½k s s þ1;k s s þ1þ s l  g K1 k¼0 ð2Þ 2.2. CGAE-based learning of the condition features In recent years, a large number of proposals based on deep learning have been reported for feature extraction and representation learning tasks. Their application fields are as diverse as the different architectures proposed for each specific task. In general, however, their power lies in some hierarchical feature extraction process through the stacking of specialized computational layers for pattern recognition and classification. Although the idea of representing learning through a feature space transformation was conceived previously to Deep Learning and several models from the traditional machine learning use this technique (for example, SVM model and MLP model), one of the credits of Deep Learning is to focus attention on this mechanism as a means of improving models only limited by the available computational resources. In the context of fault diagnosis applications, the efforts are focused on learning the transformation providing the better representation of input signals. For the i-th sensor, we can write this transformation as: x i ¼Encðs i ðtÞ;/Þ ð3Þ where Encð;/Þrepresents the learned transformation parameterized by /, also called encoder, and x i is the obtained representation for s i ðtÞ. With a good representation, building a condition monitoring model can be reduced to approximate the unknown conditional probability distribution of a failure condition ygiven the representation x i ;PðYjX i Þ, and then the resulting failure is selected through a h-parameterized model with the following criteria: y¼argmax y Pðyjx i ;hÞ ð4Þ If we denote by D i the training data set composed by tuples ðs i ;yÞfrom all the 3D printer conditions to be identified in the i-th sensor, the maximum likelihood estimation (MLE) approach can be used for finding h. Combining this with Eq. (3), we can write: h¼argmax h Y ðs i ;yÞ2D i PðyjEncðs i ;/Þ;hÞ ð5Þ Several proposals have been introduced for tuning /, as autoencoders or Boltzmann machines. In all cases, the encoder is unsupervisedly pre-trained (without using y m ) and then it is fine-tuned for the specific classification task through the MLE criterion. However, this approach cannot be applied for fault detection in real applications because the size of D i for some of the different conditions of the printer is minimal, or even non-existing, during the building phase of the model. Usually, only a large number of examples from normal condition are available in a real situation. With this in mind, we present a CGAE model for representing a useful transformation for the fault detection of 3D printers, which is learned only from signals in normal (healthy) condition. It consists of two 1DCNN and one 1D Transpose-Convolutional Neural Network (1DTCNN), both able to deal directly with input time-series with a reduced number of parameters, which decrease the risk of overfitting. 1DCNN is a model composed of a set of convolutional layers, where for a layer lthere is an input in the form of a multichannel time-series, ~ s i l ðtÞ2R TN c , to which a set of 1D convolution filters, h i l ðtÞR W;N c ;N o , is applied to produce an output multichannel time-series, o i l ðtÞ2R T o N o , as follows: o i;n o l ðtÞ¼X N c n c ¼1 ~ s i;n c l ðtÞh i;n c ;n o l ðtÞþb i;n o l ;8n o ¼1;...;N o ð6Þ where ~ s i;1 1 ðtÞ¼~ s i ðtÞ, and T;T o ;N c ;N o and Wstands for the length of input signal, length of output signal, number of input channels, number of required output channels after the convolution operation, and length of 1D convolution filters, respectively, b i;n o l is a bias term applied to adjust an extra displacement in case to be required, and is the convolution operator with the intrinsic parameters padding and stride that maintains the input length and downsamples the input signal [38]. All these parameters depend on land can be different in each layer. The input ~ s i l ðtÞis obtained from the application of batch normalization (BN) [39] to s i l ðtÞ, as follows:  s i l ðtÞ¼s i l ðtÞ l ffiffiffiffiffiffiffiffiffiffiffiffiffiffi r 2 þ pð7Þ ~ s i l ðtÞ¼ g  s i l ðtÞþbð8Þ where l and r 2 stands for the mean and variance of the signal batch in training time, respectively (or an estimation of these values in testing time), and g and bare the scaling and shifting parameters, which are estimated in training time.  is a constant close to zero, which is added to avoid the indetermination when r 2 !0. BN is applied to the inputs of each convolutional layer except the last one to avoid adding extra variance to the output of 1DCNN. After that, a nonlinear element-wise function fðÞ is applied to the output of Eq. (6): ~ s i;n c lþ1 ðtÞ¼f½o i;n o l ðtÞ ð9Þ where we must take into account that the N o output channels from l-th layer are the N c input channels for ðlþ1Þ-th layer, and T o can be computed from padding and stride parameters as: T o ¼ TW stride þ1;if padding ¼ v alid T stride ;if padding ¼same (ð10Þ where padding ¼same states for adding samples to the input signal for keeping T o ¼T(only when stride ¼1), and padding ¼ v alid states when no sample is added and the output length is mandatorily decremented. By controlling the stride parameter, the downsampling of the signal is modified in each layer. This process results in features with a high abstraction level and less time-dependency when more convolutional layers are stacked. On the other hand, 1DTCNN model performs the inverse process of 1DCNN. It uses the transpose-convolutional layer to up-sampling the input features until they are transformed into time-series. Let H2R T o T o be the orthonormal transformation matrix built with a hfilter such that: ~ z¼H~ sð11Þ where ~ zis an input channel to a transpose-convolutional layer, and it is truth that ~ z¼h~ s. Then, due to the orthonormality of H, the transpose convolution is simply defined as: ~ s¼H > ~ zð12Þ Therefore, the transpose-convolution operation over a set of input features ~ z i l 2R T;N c and output of the l-th transposeconvolutional layer can be defined as: o i;n o l ðtÞ¼X N c n c ¼1 ðH i;n c ;n o l Þ > ~ z i;n c l þb i;n o l ð13Þ ~ z i;n c lþ1 ðtÞ¼g½o i;n o l ðtÞ ð14Þ ~ z i;1 1 ¼~ z i ð15Þ Again, ~ z i;n c l is the result of batch normalization over z i;n c l , and a nonlinear function, gðÞ, is applied to o i l 2R T o ;N o to obtain the output. As before, the output of the l-th layer is the input for the next one. As mentioned above, the purpose of using 1DTCNN layers in our fault detection context is to transform from a feature space to the sensor time-series space. Then, typically T o >Tand it can be computed according to the same parameters, padding and stride as follows: T o ¼ðT1Þstride þW;if padding ¼ v alid T stride;if padding ¼same ð16Þ The training process of the CGAE model is based on BiGAN as proposed in [40]. It is a 1DCNN built for approximating an optimum vector code z i P Enc given s i P R signal input, i.e x i ¼Encðs i Þ. Our hypothesis states that if the model is trained only with normal condition examples (with P R probability distribution), then the resulting codes for signals with similar distribution will occupy a closer place in the feature space than signals measured in a failure condition of the 3D printer. Additionally to Enc, BiGAN is composed by two more models interacting with each other: a generator model (Gen), and a discriminator model (Dis). Gen is a 1DTCNN which receives z i P z , an input sampled from a known distribution, and transforms it into a signal s i P Gen .Dis is a 1DCNN which takes input tuples ðs i ;z i Þand returns its result according to the next function: Dðs i ;z i Þ¼ 1;ifs i P R and z i P Enc 0;ifs i P Gen and z i P z (ð17Þ Gen must be improved in order to fool Dis.Dis improves avoiding to be fooled by Gen and, at the same time, correctly detects the inputs from P R .Ifh Dis ;h Gen ;h Enc are the parameters of Dis;Gen and Enc, respectively, the cross-entropy loss for its optimization can be determined as: Lðh Dis ;h Gen ;h Enc Þ¼X B j¼1 logP Dis ðy j js i j ;z i j Þ ð18Þ 4C. Li et al. / Mechanical Systems and Signal Processing 147 (2021) 107108 where y j js i j ;z i j is a known example in the data batch (with size B) with y j ¼1ifs i j P R and z i j P Enc , and y j ¼0 otherwise. P Dis distribution can be described as a Bernoulli distribution, and consequently from Eq. (18) we obtain: Lðh Dis ;h Gen ;h Enc Þ¼X B j¼1 logð½Disðs i j ;z i j Þ y j ½1Disðs i j ;z i j Þ 1y j Þð19Þ ¼X B j¼1 log½Disðs i j ;z i j Þ y j þX B j¼1 log½1Disðs i j ;z i j Þ 1y j ð20Þ ¼X B j¼1 y j logDisðs i j ;z i j ÞþX B j¼1 ð1y j Þlog½1Disðs i j ;z i j Þ ð21Þ Summarizing as expectations: Lðh Dis ;h Gen ;h Enc Þ¼E s i P R logDisðs i ;Encðs i ÞÞþE z i P z log½1DisðGenðz i Þ;z i Þ ð22Þ Therefore, the improvement of the models can be established as the next minimax optimization problem: h Dis ;h Gen ;h Enc ¼argmin h Gen ;h Enc argmax h Dis Lðh Dis ;h Gen ;h Enc Þð23Þ The optimization (training stage) of Eq. (23) can be performed through the algorithm described in [41] for GAN-based models. The application of this procedure to each D i signal set (from i-th sensor) results in one Enc i model capturing the behavior of each i-th sensor under normal condition of the machine. 2.3. Deep encoded features fusion through SVM Taking only the encoder into account, the optimization of Eq. (23) returns a set of improved deep convolutional networks able to map signals to feature spaces. The next step is to combine those spaces of independent features for each sensor into one that is enriched by the group of sensors. This task will be performed by concatenating the outputs of each Enc i model, i.e.: X¼½Enc 1 ðÞ;Enc 2 ðÞ;Enc 3 ðÞ;Enc 4 ðÞ;Enc 5 ðÞ;Enc 6 ðÞ ð24Þ where Enc i ðÞ is a 1DCNN specialized in the i-th sensor and Xis the enriched feature space. For the creation of a decision function, firstly the samples from D i are projected into Xto build the set: D¼[ j Enc i ðs i j ðtÞÞ hi 6 i¼1 ð25Þ Now, a binary decision function that separates the region of the input space containing instances with the distribution of the examples from Dcan be created as an alternative solution to the problem posed in Eq. (4). To this end, we propose the use of a SVM-based approach. According to [42],f SVM can be defined as: f SVM ðxÞ¼sgn X j a j kðx j ;xÞ q !ð26Þ where sgn takes values f1;0;1g, a j is the Lagrange multiplier associated with the support vector x j ( a j >0) taken from the set D, and q is the offset factor of the decision hyperplane in a projection space determined by RBF-kernel: kðx;yÞ¼e  c jjxyjj 2 ð27Þ where c is the length scale parameter determining the influence radius in the enriched feature space of the support vectors. The set of coefficients a of Eq. (26) can be computed using the method developed in [43] by solving the next optimization problem: a ¼argmin a 1 2X j X j 0 a j a j 0 kðx j ;x j 0 Þ;s:t:06 a j 61 m jDj;X j a j ¼1ð28Þ where m is the upper bound of abnormal examples in D. And then q can be estimated by: q ¼X j 0 a j kðx j 0 ;x j Þ ð29Þ where x j is any support vector associated with non-upper/lower bound Lagrange multiplier. Finally, m and c are hyperparameters of the model that will be estimated according to the methods detailed in Section 3.2 or 3.3. 2.4. Overview of the proposed fCGAE approach for 3D printer fault detection After obtaining the structure of fCGAE (Fig. 1 shows a summary of the process), it is necessary to train the model. Enc 1 ;...;Enc 6 and f SVM can be trained separately, but the outputs from each Enc i model must be combined into f SVM for data fusion training. The application of fCGAE for fault detection of 3D printers can be summarized as follows: Step 1. Collect s 1 ðtÞ;...;s 6 ðtÞfrom 3-axis velocity sensors and 3-axis angle sensors under normal condition in the 3D printer (one given task). Step 2. Compute augmented signal sets D 1 ;...;D 6 using Eq. (2). Step 3. Build Enc 1 ;...;Enc 6 ;Gen 1 ;...;Gen 6 and Dis 1 ;...;Dis 6 according to Eqs. (6), (9) (for Enc and Dis), and Eqs. (13), (14) (for Gen), and take as inputs s 1 ðtÞ;...;s 6 ðtÞ;Gen 1 ðz 1 Þ;...;Gen 6 ðz 6 Þand ðs 1 ;z 1 Þ;...;ðs 6 ;z 6 Þ, respectively. Step 4. From the outputs of Enc 1 ;...;Enc 6 , build f SVM for data fusion using the input given by Eq. (24). Step 5. Train Enc 1 ;...;Enc 6 models separately by optimizing Eq. (22), and complete fCGAE training by feeding the results to f SVM for its optimization. Step 6. Apply trained fCGAE for fault detection by entering new velocity and angle signals into it to obtain the machine condition. 3. Hyperparameters search Two groups of hyperparameters need to be identified: those from the networks architecture, and those from OCSVM model. For the first group, we propose an heuristic-based approach for minimizing the amount of information lost between layers. For the second group, we evaluate two approaches; (i) cross-validated random search, and (ii) unsupervised search methods. Details are presented in the next paragraphs. 3.1. Networks architecture The number of layers (L), number of output channels in every layer (fN l o g), kernel lengths (fW l g), stride and padding define the architecture of Enc;Dis, and Gen networks. Their configuration follows a principle of minimum information lost between the network layers: given a raw signal of length Tand configuring stride ¼2 as the minimum downsampling for having translation invariance in each layer, Lcan be computed for the three networks as: L¼blog 2 Tc ð30Þ As the selected stride decreases the signal length to the half, N l o compensates this information loss, i.e., for Enc and Dis: Fig. 1. Structure of fCGAE model for fault detection in a 3D printer. fN l o ¼2 l g L1 l¼1 ð31Þ and for Gen: fN l o ¼2 Ll g L l¼2 ð32Þ Setting Enc and Gen inverse architectures, N L o of Enc and N 1 o of Gen are both the same as the dimension of the desired feature space. Furthermore, N L o ¼1 for Dis, as requires an adversarial-based approach. If T l in is the input signal length of the lth layer in Enc and Dis, then padding ¼same if T l in is even and l<L, and padding ¼ v alid otherwise. If W max is the maximum kernel length, then the W l values for Enc and Dis must verify: if padding ¼same and W max can be applied at least 2 times in T l in , then W l ¼W max . if padding ¼ v alid, then W l ¼W max 1. It produces an even input for the next layer. In the last layer, W L ¼T L in for computing the last features using the entire input. W l value of Gen is equal to W L lþ1 value of Enc, and the same inverse configuration must be applied for the padding parameter. 3.2. Cross-validated random search Contrarily to grid search [44], Cross-Validated Random Search method (CVRS) evaluates only Ncandidates sampled from the hyperparameter search space. This method highly reduces the computational cost of the search process with almost similar results compared with the grid search in Deep Learning applications [45]. CVRS can optimize m and c parameters of our approach and also hyperparameters of other machine learning-based proposals. However, it is necessary to point out that CVRS assumes a calibration set available in the training stage with abnormal examples to evaluate the detection model. CVRS requires a metric to evaluate the hyperparameter set quality. We choose the balanced accuracy since it considers the unbalance that could have the evaluation set in each iteration of the CVRS. Let tp;tn;fp and fn be the true positives, true negatives, false positives, and false negatives, respectively; the balanced accuracy for a binary classifier as f SVM ðÞ is defined by: balanced accuracy ¼ tp tpþfn þ tn tnþfp 2ð33Þ Then, the CVRS procedure adapted for the one-class learning problem is: 1. Select Ncandidate sets of the search space according to the probability distribution of the hyperparameters. 2. Split the training and calibration sets into Kpartitions. 3. For n¼1toN: (a) Configure the model with the n-th hyperparameter set. (b) For k¼1toK: i. Train the model with the other K1 partitions (all but the k-th partition) of the training set. ii. Evaluate the model in the k-th partition of the training and calibration sets. (c) Compute the average balanced accuracy over the Kprevious results for the n-th hyperparameter set. 4. Choose the hyperparameter set with the highest average balanced accuracy. 5. Train a model with the entire training set and chosen hyperparameters. 6. Evaluate the model in the test set. 3.3. Unsupervised search methods As it was stated before, CVRS requires a calibration set that could be difficult to have in the training stage. This restriction could be lightened if the cross-validation process uses a fixed calibration set for every iteration; in this case, the calibration set size is smaller than the training set (for K¼10, the calibration set size is 1=10 of the training set). In this work, we also tackle the worst case where a calibration set is not available for the hyperparameters search. This issue is addressed with the approaches proposed by Xiao et al. [46] and Ratsch et al. [47] for tuning c and m with only the one-class examples in the training set. That is, if fx i g m 1 be a one-class training set with mexamples, then c can be optimized maximizing the next objective function: fð c Þ¼2 mX m i¼1 e  c min j–ijjx i x j jj 2 2 mX m i¼1 e  c max jjjx i x j jj 2 ð34Þ The fð c Þfunction can be maximized with any optimization numerical method of scalar functions. In this work, we used Brent’s method [48]. The m parameter optimization follows an interactive procedure to maximize the projected distance between the examples classified as þ1 and 1byf SVM . Let inc ¼ 1 N nu the m increment of each iteration splitting the interval in N nu homogeneous steps, then the procedure is as follow: 1. For m ¼inc to 1 inc, in steps of Inc.: (a) Train OCSVM with the optimized c . (b) Compute the average distance to the hyperplane of the examples with f SVM ¼þ1. (c) Compute the average distance to the hyperplane of the examples with f SVM ¼1. (d) Compute the total distance of the two groups by adding the previous distances. 2. Choose m that maximizes the total distance. 4. Experiments In this section a 3D printer experimental setup is built to collect 3-axis velocity and 3-axis angle measurements and to test the proposed fCGAE technique to fault detection. Comparison with peer approaches are also introduced at the end of the section. 4.1. Delta 3D printer test bed We tested the performance of this model on the experimental platform for 3D printers developed by our group. The 3D printer (SLD-BL600-6, brand SHILEIDI) to be diagnosed has a delta kinematic configuration with 3 degrees of freedom (Fig. 2). A bipolar step motor each join, which provides movement to a synchronous belt-driven transmission mechanism for the arm displacement. Each arm is composed of two metal rods coupled at their ends by universal join bearings. A low-cost attitude sensor BWT901 type MEMS (brand WIT), installed at the base of the delta configuration, collects the 3-axis velocity and angle signals. This device includes an analog-to-digital conversion system with a sensitivity in the velocity measurement offered by the sensor manufacturer of 0:05 deg =s and 0:01 deg for the angular measurement. The sampling frequency was set at 100 Hz with time synchronization for both types of measurements. Digital samples were collected with a USB interface. The signals were obtained under detailed conditions for every case study and captured in the computer while the base of the printer performs a circular movement of 75 mm radius with 20 repetitions in a total time of 324 s. The whole process was repeated 3 times with a uniformly distributed random starting time between 1 and 60 s after finishing the previous signal acquisition. Therefore, we obtained 3 multidimensional time series (3 dimensions for velocity and 3 for angle) with a duration of 32400 samples each. Later, for the practical application and evaluation of our test in a realistic scenario, it has to be performed at a trial stage where the machine makes one circular movement providing data to the model. This trial stage could be included before and/ or after the printing working to avoid disturbances during the device operation. A trial-before allows detecting faults that could affect the printing quality. While a trial-after allows detecting faults resulting from a printing process. However, other trial stages could also be implemented in the middle of extensive printing workings, if possible. The real-time monitoring of the 3D printers under a variable operation condition is out of the scope of this work. 4.1.1. Case study 1: Join bearings A 3D printer fault causes generally a loss in the final product quality. However, the acceptable production tolerances, which are related with the fault severity, depends on the specific application in which the product will be used. In this work, we empirically determined the severity by increasing the fault and inspecting signs of deterioration in the surface, corners and edges of the product. The previous procedure stops when the first signs of quality change appear. Therefore, the resulting failures can be considered as early-stage faults. The first case study is about join bearing fault detection caused by loosening in the fastening screws, commonly caused by prolonged use of the machine. Products resulting from 3D printers with this failure present surface disturbances as shown in Fig. 3. Faults can occur in any of the rods associated with an arm, as well as in any of its ends. In this way, 12 faults (2 rods 2 ends 3 arms) of this type could be configured (plus the normal condition, named P) according to their location as shown in Table 1. Each failure was artificially created by loosening of 0.7 mm in the screw (2 turns). 4.1.2. Case study 2: Belts The second case study is fault detection in synchronous belts. This type of failure is also due to long periods of use of the machine, causing an additional stretch in the material of the belts. It produces additional oscillations when high acceleration braking is required, which results in a final product of low quality detail. To obtain a dataset under this type of failure, 1.5 mm of belt was loosened from its optimal clamping position as shown in Fig. 4. Three fault conditions were obtained, one for each synchronization belt, named as M;Nand Ofor arm 1, 2 and 3, respectively. As in the previous case, the normal condition of the machine is also available. 4.2. Experimental setup for comparisons From the three multidimensional time series with length 32400 obtained under normal condition of the printer, two of them were used for training and one for testing. Then, the process described in SubSection 2.1 was applied to increase the size of the datasets with s l ¼1620 (32400/20) to obtain time series corresponding to a complete circle, and s s ¼10. From this, a training dataset with 6158 examples and a test dataset with 3079 examples are obtained under normal condition of the machine. Similarly, two signals of each fault condition were used to create the calibration set and one for the test set after applying them the same procedure. 4.2.1. Feature extraction The models to be compared to our approach have difficulties in dealing with common representations of signals in domains such as time, frequency and/or time–frequency, therefore, as it was proposed in the methodology presented in [49], the crest factor, shape factor, absolute average amplitude, square root amplitude, kurtosis, variance, clearance factor, impulse indicator, and skewness factor were instead used as condition indicators for each of the 6 available signals in every example represented in the three domains. There are 32 sub-signals obtained through the wavelet packet decomposition procedure until level 5 for the time–frequency representation, and nine condition indicators were obtained from each of these sub-signals. This results in a features vector of length 9 for time, same for frequency and, finally, another of length 288 for time–frequency domain, resulting in a vector of length 306 for each signal. Then, six vectors (one for each channel of the sensor) were fused into one with 1836 features to build a general representation of each available sample in the dataset. Fig. 2. Test bed of the 3D printer. such as 59:89%in the Econdition. This indicates that SVM can generate better discrimination functions in complex representation spaces. 5.3. Performance for case study 2 Tables 12 and 13 show the detection rate for models based on SVM and iForest, respectively, for case study 2. Again, the best results are obtained by combining the two types of signals (six channels), independently whether SVM or iForest is used. The disadvantage of using only velocity signals (fCGAE1 and fCGAE1F) becomes more evident, where the performance decreases for the detection of all the faults of the case study. These results present even worse performance than autoencoder based models, that used features extracted from both velocity and angle signals. As previously, fCGAE3u has an excellent performance considering the data restrictions it was created with. It is in third position in the best performances, surpassing all iForest-based models and only behind of fCGAE3 and fCGAE2. To sum up, overall performance for the two study cases gives advantage to fCGAE3. Therefore, greater generalization capacity is appreciated for models that use SVM. Results from dAEForest and sAEForest in conjunction with dAESVM and sAESVM for both cases show that the obtained feature space is not informative enough to detect some faults for 3D printer. fCGAE3u gives a useful alternative in the worse data restriction scenario. 6. Conclusions This work has proposed a new method for the construction of a fault detection model for 3D printers from the fusion of 3- axis velocity and angle signals obtained exclusively from normal condition. After the signal acquisition phase, a procedure for increasing the number of examples was applied over the healthy training dataset. With this augmented data, the construction of six 1DCNN encoder models was carried out with a BiGAN-based approach, and their outputs were fused into an OCSVM model optimized to create a normal condition profile with the available data. The resulting model was able to correctly discriminate new examples as healthy or faulty condition. The proposed approach has been evaluated on two cases in 3D printers: fault detection of 12 different join bearings, and fault detection in 3 synchronous bands. Additionally, the method was compared with multiple methods reported in literature for anomaly detection, and also with the resulting models without fusing different type signals. The results allow us to conclude the following points: 1DCNN type encoders created with a GAN-based approach allow to build a deep feature space from raw signals without any pre-processing that is informative enough for the fault detection task, surpassing feature engineering-based approaches and autoencoders. The feature space created by the encoders can be used by models that characterize the normal condition (SVM) as well as by models that characterize the anomaly (iForest), with acceptable results in both cases. The fusion of multi-channel signals gives the resulting model a better generalization capacity for the detection of various faults, compared to that obtained from models of a single type of signals. A detection model with excellent performance can be created without a calibration dataset using the proposed heuristic for the network architectures together unsupervised approaches for tuning hyperparameters. As future work, the exploration of the feature space is proposed for the identification of the type of failure and the possible assessment of its severity. Also, although this work is devoted to including an additional trial stage for detecting faults, which is feasible in most of the 3D printing applications, the extension of the proposed method to real-time 3D printer monitoring is an interesting topic to be explored in the future. This extension process should consider the intrinsic complexity added by non-stationary time series measured in a 3D printer working under variable operating conditions. Table 11 Success rate of iForest-based models applied to joint bearing faults. Condition fCGAE1F fCGAE2F fCGAE3F iForest dAEForest sAEForest P 87.40 98.80 99.58 94.84 72.39 73.69 A 100.0 100.0 100.0 100.0 98.67 96.07 B 94.80 100.0 100.0 100.0 73.47 57.29 C 99.97 6.72 91.43 100.0 80.77 62.68 D 99.32 100.0 100.0 100.0 95.78 87.33 E 100.0 100.0 100.0 59.89 78.17 60.41 F 96.30 100.0 99.06 100.0 51.19 64.05 G 87.11 100.0 100.0 84.77 82.10 68.40 H 78.17 100.0 100.0 100.0 94.74 82.59 I 78.89 86.72 68.30 97.43 90.16 82.04 J 92.17 100.0 100.0 98.57 90.58 72.23 K 86.29 100.0 100.0 90.81 67.26 53.07 L 93.57 100.0 100.0 99.61 57.65 43.46 CRediT authorship contribution statement Chuan Li: Supervision, Writing - review & editing, Resources, Conceptualization, Funding acquisition. Diego Cabrera: Conceptualization, Methodology, Software, Validation, Investigation, Writing - original draft. Fernando Sancho: Supervision, Writing - review & editing, Conceptualization. René-Vinicio Sánchez: Resources, Funding acquisition, Writing - review & editing, Project administration. Mariela Cerrada: Resources, Funding acquisition, Writing - review & editing, Project administration. Jianyu Long: Resources, Funding acquisition, Writing - review & editing, Project administration. José Valente de Oliveira: Conceptualization, Writing - review & editing. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgements The work was sponsored in part by GIDTEC Research Group of Universidad Politécnica Salesiana, the National Natural Science Foundation of China (51775112, 71801046), the National Key R&D Program (2016YFE0132200), the MoST Science and Technology Partnership Program (KY201802006), the Chongqing Natural Science Foundation (cstc2019jcyj- zdxmX0013), and the CTBU Project (KFJJ2018107, KFJJ2018075). References [1] T. Sathish, M. Vijayakumar, A.K. Ayyangar, Design and fabrication of industrial components using 3d printing, Materials Today: Proceedings 5 (2018) 14489–14498. [2] Z. Zahedi-Tabar, S. Bagheri-Khoulenjani, H. Mirzadeh, S. Amanpour, 3d in vitro cancerous tumor models: Using 3d printers, Medical Hypotheses 124 (2019) 91–94. [3] B. Pérez, H. Nykvist, A.F. Brøgger, M.B. Larsen, M.F. Falkeborg, Impact of macronutrients printability and 3d-printer parameters on 3d-food printing: A review, Food Chemistry 287 (2019) 249–257. [4] M. Yampolskiy, A. Skjellum, M. Kretzschmar, R.A. Overfelt, K.R. Sloan, A. Yasinsac, Using 3d printers as weapons, International Journal of Critical Infrastructure Protection 14 (2016) 58–71. [5] C. Balletti, M. Ballarin, F. Guerra, 3d printing: State of the art and future perspectives, Journal of Cultural Heritage 26 (2017) 172–182. [6] T.D. Ngo, A. Kashani, G. Imbalzano, K.T. Nguyen, D. Hui, Additive manufacturing (3d printing): A review of materials, methods, applications and challenges, Composites Part B: Engineering 143 (2018) 172–196. [7] A. Bewoor, S. Kulkarni, Interoperability of international standards, condition monitoring methods and research models for bearing fault: an integrated approach, Procedia Manufacturing 22 (2018) 982–989. [8] A. Khadersab, S. Shivakumar, Parametric vibration analysis of rotating machinery, Materials Today: Proceedings 5 (2018) 25688–25696. [9] Y. Lu, Y. Wang, Monitoring temperature in additive manufacturing with physics-based compressive sensing, Journal of Manufacturing Systems 48 (2018) 60–70. [10] B. Li, L. Zhang, L. Ren, X. Luo, 3d printing fault detection based on process data, in: Proceedings of 2018 Chinese Intelligent Systems Conference, Springer, Singapore, 2018, pp. 385–396. [11] J.K. Yoon, D. He, B.V. Hecke, A phm approach to additive manufacturing equipment health monitoring, fault diagnosis, and quality control, in: Annual Conference of the Prognostics and Health Management Society, 2014. [12] K. He, Z. Yang, Y. Bai, J. Long, C. Li, Intelligent fault diagnosis of delta 3d printers using attitude sensors based on support vector machines, Sensors18 (2018) 1298. [13] S. Zhang, K. He, D. Cabrera, C. Li, Y. Bai, J. Long, Transmission condition monitoring of 3d printers based on the echo state network, Applied Sciences 9 (2019) 3058. [14] D. Xiang, L. Jianyu, L. Chuan, C. Diego, Z. Shaohui, Intelligent fault diagnosis of 3d printers based on reservoir computing, International Journal of Performability Engineering 15 (2019) 3171. Table 12 Success rate of SVM-based models applied to belt faults. Condition fCGAE1 fCGAE2 fCGAE3 fCGAE3u OCSVM dAESVM sAESVM P 94.02 95.81 92.82 90.16 75.38 80.55 78.56 M 14.16 100.0 100.0 100.0 100.0 79.05 80.68 N 1.20 92.30 100.0 100.0 100.0 87.04 88.37 O 2.34 100.0 100.0 100.0 100.0 100.0 100.0 Table 13 Success rate of iForest-based models applied to belt faults. Condition fCGAE1F fCGAE2F fCGAE3F iForest dAEForest sAEForest P 70.09 90.19 90.09 89.90 79.96 82.14 M 31.83 100.0 100.0 95.88 69.99 70.87 N 22.15 95.41 99.97 100.0 80.09 75.35 O 22.09 100.0 100.0 100.0 100.0 100.0 [15] J. Long, Y. Hong, S. Zhang, D. Cabrera, J. Zhong, Improving extreme learning machine by a level-based learning swarm optimizer and its application to fault diagnosis of 3d printers, International Journal of Performability Engineering 15 (2019) 2972. [16] S. Zhang, Z. Sun, C. Li, D. Cabrera, J. Long, Y. Bai, Deep hybrid state network with feature reinforcement for intelligent fault diagnosis of delta 3-d printers, IEEE Transactions on Industrial Informatics 16 (2020) 779–789. [17] J. Long, S. Zhang, C. Li, Evolving deep echo state networks for intelligent fault diagnosis, IEEE Transactions on Industrial Informatics (2019), 1–1. [18] J. Long, Z. Sun, C. Li, Y. Hong, Y. Bai, S. Zhang, A novel sparse echo autoencoder network for data-driven fault diagnosis of delta 3-d printers, IEEE Transactions on Instrumentation and Measurement (2019) 1–10. [19] J. Guo, J. Wu, Z. Sun, J. Long, S. Zhang, Fault diagnosis of delta 3d printers using transfer support vector machine with attitude signals, IEEE Access 7 (2019) 40359–40368. [20] S. Zhang, Z. Sun, J. Long, C. Li, Y. Bai, Dynamic condition monitoring for 3d printers by using error fusion of multiple sparse auto-encoders, Computersin Industry 105 (2019) 164–176. [21] Z. Qiao, Y. Lei, N. Li, Applications of stochastic resonance to machinery fault detection: A review and tutorial, Mechanical Systems and Signal Processing 122 (2019) 502–536. [22] Z. Li, B. Shi, A piecewise nonlinear stochastic resonance method and its application to incipient fault diagnosis of machinery, Chinese Journal of Physics 59 (2019) 126–137. [23] X. Zhang, J. Wang, Z. Liu, J. Wang, Weak feature enhancement in machinery fault diagnosis using empirical wavelet transform and an improved adaptive bistable stochastic resonance, ISA Transactions 84 (2019) 283–295. [24] Z. German-Sallo, G. Strnad, Machinery fault diagnosis using signal analysis, Procedia Manufacturing 32 (2019) 585–590. [25] J. Li, X. Yao, H. Wang, J. Zhang, Periodic impulses extraction based on improved adaptive VMD and sparse code shrinkage denoising and its application in rotating machinery fault diagnosis, Mechanical Systems and Signal Processing 126 (2019) 568–589. [26] L. Wang, Z. Liu, Q. Miao, X. Zhang, Time–frequency analysis based on ensemble local mean decomposition and fast kurtogram for rotating machinery fault diagnosis, Mechanical Systems and Signal Processing 103 (2018) 60–75. [27] C. Li, J.L.V. de Oliveira, M.C. Lozada, D. Cabrera, V. Sanchez, G. Zurita, A systematic review of fuzzy formalisms for bearing fault diagnosis, IEEE Transactions on Fuzzy Systems (2018), 1–1. [28] C. Li, M. Cerrada, D. Cabrera, R.V. Sanchez, F. Pacheco, G. Ulutagay, J.V. de Oliveira, A comparison of fuzzy clustering algorithms for bearing fault diagnosis, Journal of Intelligent & Fuzzy Systems 34 (2018) 3565–3580. [29] D. Cabrera, F. Sancho, C. Li, M. Cerrada, R.-V. Sánchez, F. Pacheco, J.V. de Oliveira, Automatic feature extraction of time-series applied to fault severity assessment of helical gearbox in stationary and non-stationary speed operation, Applied Soft Computing 58 (2017) 53–64. [30] R. Liu, B. Yang, E. Zio, X. Chen, Artificial intelligence for fault diagnosis of rotating machinery: A review, Mechanical Systems and Signal Processing 108 (2018) 33–47. [31] D. Cabrera, F. Sancho, M. Cerrada, R.-V. Sánchez, F. Tobar, Echo state network and variational autoencoder for efficient one-class learning on dynamical systems, Journal of Intelligent & Fuzzy Systems 34 (2018) 3799–3809. [32] S. Plakias, Y.S. Boutalis, Exploiting the generative adversarial framework for one-class multi-dimensional fault detection, Neurocomputing 332 (2019) 396–405. [33] H. Liu, J. Zhou, Y. Xu, Y. Zheng, X. Peng, W. Jiang, Unsupervised fault diagnosis of rolling bearings using a deep neural network based on generative adversarial networks, Neurocomputing 315 (2018) 412–424. [34] J. Saari, D. Strömbergsson, J. Lundberg, A. Thomson, Detection and identification of windmill bearing faults using a one-class support vector machine, SVM, Measurement 137 (2019) 287–301. [35] M. Zeng, Y. Yang, S. Luo, J. Cheng, One-class classification based on the convex hull for bearing fault detection, Mechanical Systems and Signal Processing 81 (2016) 274–293. [36] P.-P. Xi, Y.-P. Zhao, P.-X. Wang, Z.-Q. Li, Y.-T. Pan, F.-Q. Song, Least squares support vector machine for class imbalance learning and their applications to fault detection of aircraft engine, Aerospace Science and Technology 84 (2019) 56–74. [37] Z. Cui, W. Chen, Y. Chen, Multi-scale convolutional neural networks for time series classification, arXiv:1603.06995v4 (2016). [38] V. Dumoulin, F. Visin, A guide to convolution arithmetic for deep learning, arXiv:1603.07285v2 (2018). [39] S. Ioffe, C. Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift, arXiv:1502.03167v3 (2015). [40] J. Donahue, P. Krähenbühl, T. Darrell, Adversarial feature learning, arXiv:1605.09782 (2017). [41] I.J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial networks, arXiv:1406.2661v1 (2014). [42] B. Schölkopf, R. Williamson, A. Smola, J. Shawe-Taylor, J. Platt, Support vector method for novelty detection, (2000). [43] B. Schölkopf, J.C. Platt, J. Shawe-Taylor, A.J. Smola, R.C. Williamson, Estimating the support of a high-dimensional distribution, Neural Computation 13 (2001) 1443–1471. [44] J. Wainer, G. Cawley, Nested cross-validation when selecting classifiers is overzealous for most practical applications, arXiv:1809.09446v1 (2018). [45] J. Bergstra, Y. Bengio, Random search for hyper-parameter optimization, Journal of Machine Learning Research 13 (2012) 281–305. [46] Y. Xiao, H. Wang, L. Zhang, W. Xu, Two methods of selecting gaussian kernel parameters for one-class SVM and their application to fault detection, Knowledge-Based Systems 59 (2014) 75–84. [47] G. Ratsch, S. Mika, B. Scholkopf, K.-R. Muller, Constructing boosting algorithms from SVMs: an application to one-class classification, IEEE Transactions on Pattern Analysis and Machine Intelligence 24 (2002) 1184–1199. [48] R.P. Brent, An algorithm with guaranteed convergence for finding a zero of a function, The Computer Journal 14 (1971) 422–425. [49] C. Li, R.-V. Sanchez, G. Zurita, M. Cerrada, D. Cabrera, R.E. Vásquez, Multimodal deep support vector classification with homologous features and its application to gearbox fault diagnosis, Neurocomputing 168 (2015) 119–127. [50] F.T. Liu, K.M. Ting, Z.-H. Zhou, Isolation forest, in: 2008 Eighth IEEE International Conference on Data Mining, IEEE, 2008. [51] Y. Xiao, H. Wang, W. Xu, J. Zhou, Robust one-class SVM for fault detection, Chemometrics and Intelligent Laboratory Systems 151 (2016) 15–25. [52] G.A. Susto, A. Beghi, S. McLoone, Anomaly detection through on-line isolation forest: An application to plasma etching, in: 2017 28th Annual SEMI Advanced Semiconductor Manufacturing Conference (ASMC), IEEE, 2017. [53] A. Borghesi, A. Bartolini, M. Lombardi, M. Milano, L. Benini, Anomaly detection using autoencoders in high performance computing systems, arXiv:1811.05269v1 (2018). [54] P. Vincent, H. Larochelle, Y. Bengio, P.-A. Manzagol, Extracting and composing robust features with denoising autoencoders, in: Proceedings of the 25th International Conference on Machine Learning – ICML, ACM Press, 2008. [55] F. Li, J.M. Zurada, W. Wu, Sparse representation learning of data by autoencoders with regularization, Neural Network World 28 (2018) 133–147. [56] D.P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv:1412.6980v9 (2014). [57] B. Xu, N. Wang, T. Chen, M. Li, Empirical evaluation of rectified activations in convolutional network, arXiv:1505.00853v2 (2015). [58] S.L. Smith, P.-J. Kindermans, C. Ying, Q.V. Le, Don’t decay the learning rate, increase the batch size, arXiv:1711.00489v2 (2017).