Temporal convolutional networks applied to energy-related time series forecasting

applied sciences Article Temporal Convolutional Networks Applied to Energy-Related Time Series Forecasting Pedro Lara-Benítez *,† , Manuel Carranza-García †, José M. Luna-Romera and José C. Riquelme Division of Computer Science, University of Sevilla, ES-41012 Seville, Spain; [email protected] (M.C.-G.); [email protected] (J.M.L.-R.); [email protected] (J.C.R.) *Correspondence: [email protected] † These authors contributed equally to this work. Received: 4 March 2020; Accepted: 24 March 2020; Published: 28 March 2020   Featured Application: Energy demand forecasting to improve power generation management. Abstract: Modern energy systems collect high volumes of data that can provide valuable information about energy consumption. Electric companies can now use historical data to make informed decisions on energy production by forecasting the expected demand. Many deep learning models have been proposed to deal with these types of time series forecasting problems. Deep neural networks, such as recurrent or convolutional, can automatically capture complex patterns in time series data and provide accurate predictions. In particular, Temporal Convolutional Networks (TCN) are a specialised architecture that has advantages over recurrent networks for forecasting tasks. TCNs are able to extract long-term patterns using dilated causal convolutions and residual blocks, and can also be more efficient in terms of computation time. In this work, we propose a TCN-based deep learning model to improve the predictive performance in energy demand forecasting. Two energy-related time series with data from Spain have been studied: the national electric demand and the power demand at charging stations for electric vehicles. An extensive experimental study has been conducted, involving more than 1900 models with different architectures and parametrisations. The TCN proposal outperforms the forecasting accuracy of Long Short-Term Memory (LSTM) recurrent networks, which are considered the state-of-the-art in the field. Keywords: deep learning; energy demand; temporal convolutional network; time series forecasting 1. Introduction Forecasting electricity demand is currently amongst the most important challenges for the industries. Due to the increasingly high level of electricity consumption, electrical companies need to efficiently manage the production of energy. Sustainable production plans are required to meet demands and account for important challenges of this century such as global warming and the energy crisis. Smart meters now provide useful data that can help to understand consumption patterns and monitor power demand more efficiently. Data mining techniques can use this information to learn from historical past data and predict the expected demand to make decisions accordingly. Obtaining accurate forecasts can be essential for the future electricity market considering the increasing penetration of renewable energies. However, forecasting power demand is a complex task that involves many factors and requires sophisticated machine learning models to produce high-quality predictions. Statistical-based models, such as the Box–Jenkins model called ARIMA, were for many years the state-of-the-art for electricity time series forecasting [ 1 , 2 ]. However, machine learning models have proven to provide better performance for problems of this domain. Artificial neural networks Appl. Sci. 2020,10, 2322; doi:10.3390/app10072322 www.mdpi.com/journal/applsci Appl. Sci. 2020,10, 2322 2 of 17 (ANNs) [ 3 ], support vector machines (SVMs) [ 4 , 5 ], and regression trees [ 6 ] have been applied successfully for diverse power demand prediction tasks. More recently, deep learning (DL) has emerged as a very powerful approach for time series forecasting. DL models are especially suitable for big-data temporal sequences due to their capacity to extract complex patterns automatically without feature extraction preprocessing steps [7]. As an evolution from simple ANNs, deep, fully connected networks have been applied for load forecasting problems [ 8 ]. However, fully connected networks are unable to capture the temporal dependencies of a time series. Consequently, more specialised DL models such as recurrent neural networks (RNNs) and convolutional neural networks (CNNs) started to gain importance in the time series forecasting field. These networks can efficiently encode the underlying patterns of time series by transforming the temporal problem into a spatial architecture [ 9 ]. In the recent literature, a significant number of studies presenting results of the application of RNNs to energy-related time series forecasting can be found [ 10 , 11 ]. Among all existing RNN architectures, long short-term memory (LSTM) networks have been the most popular due to their capacity to solve problems of previous RNN such as gradient explosion and vanishing gradient [ 12 ]. It has been considered a standard forecasting model for several tasks such as traffic prediction [ 13 ], solar power forecasting [ 14 ], financial market predictions [ 15 ], and electricity price prediction [ 16 ]. Although CNNs were originally designed for computer vision tasks, they are also suitable for time series data since they can extract high-level features from data with a grid topology. Despite the popularity of RNNs, several works using convolutional networks can be found. In both [ 17 , 18 ], the authors proposed CNN models for short-term load forecasting that provides comparable results to LSTM models. Other works have been able to build deep convolutional networks that can outperform LSTM networks for electricity demand [ 19 ] and solar power data problems [ 20 ]. Furthermore, in all these works, the CNN models proved to be more suitable for real-time applications given their faster training and testing execution time. The properties of local connectivity and parameter sharing of convolutional networks reduce the number of trainable parameters compared to RNNs, hence they can be trained more efficiently. There have also been proposals using hybrid models that combine convolutional and LSTM layers. In [ 21 ], the output feature maps of a CNN are fed to a RNN that provides the prediction. Other approaches consider combining the features extracted in parallel from a CNN and a LSTM to improve the forecasting using electricity demand data [ 22 ] or financial data [ 23 ]. These ensemble proposals can enhance the predictive performance by fusing the long-term patterns captured by the LSTM and the local trend features obtained with the CNN. More recently, a specialised CNN architecture known as temporal convolutional networks (TCN) has acquired popularity due to their suitability to deal with time series data. TCNs were first proposed in [ 24 ], in which they were compared to several RNNs over sequence modelling tasks. TCNs use causal dilated causal convolution in order to be able to capture longer-term dependencies and prevent information loss. Furthermore, they present other advantages over RNNs such as lower memory requirements, parallel processing of long sequences as opposed to the sequential approach of RNNs, and a more stable training scheme. Several works have already successfully used TCNs for time series forecasting tasks: the original architecture using stacked dilated convolutions was proposed in [ 25 ] to improve the performance of LSTM networks for financial domain problems; Ref. [ 26 ] designed a deep TCN for multiple related time series with an encoder–decoder scheme, evaluating over data from the sales domain; the study in [ 27 ] proposed a multivariate time series forecasting model for meteorological data, which outperformed several popular deep learning models. However, to the best of our knowledge, the potential of TCNs has not yet been explored for univariate time series forecasting problems related to electricity demand data. In this work, we study the applicability and performance of TCNs for multistep time series forecasting over two energy-related datasets. With the first dataset, we build a deep learning model to forecast the electricity demand in Spain based on the historical consumption data over five years. In the second dataset, the problem is to forecast the expected energy consumption of charging stations for electric vehicles in Spain. Our aim in this study is to present a deep learning model that uses a TCN Appl. Sci. 2020,10, 2322 3 of 17 to obtain high accuracy on time series forecasting. We present the results obtained with several TCN architectures and perform an extensive comparison with different LSTM models, which has been so far the most extended approach for these types of problems. In the experimental study, we carry out an extensive parameter search process which involves 1998 different network architectures. In summary, the main scientific contributions of this paper can be condensed as follows: • A temporal convolutional neural network model to achieve high accuracy in forecasting over energy demand time series; • A thorough experimental study, comparing the performance of temporal convolutional with long short-term memory networks for time series forecasting. The rest of the paper is organised as follows: Section 2describes the materials used, the methodology, and the experiments carried out; in Section 3, the experimental results obtained are reported and discussed; Section 4presents the conclusions and future work. 2. Materials and Methods In this section, we present the datasets selected for the study, the methodology to perform time series forecasting using deep learning models, and the details of the experimental study carried out. 2.1. Datasets In this subsection, we present the two energy-related datasets selected for the study. 2.1.1. Electric Demand in Spain The first dataset used in the experimental study covers the national electrical energy demand in Spain ranging from 2014-01-02T00:00 to 2019-11-01T23:50. During this period, the consumption was measured every ten minutes which makes a time series with a total length of 306,721 measurements. The data was provided by Red Eléctrica de España (the Spanish public grid) and is publicly available at [ 28 ]. The dataset was divided into two sets: the training set contains 245,376 samples corresponding to the period from 2014-01-02T00:00 to 2018-09-02T00:50; and the test set contains 61,343 samples comprising the period from 2018-09-02T01:00 to 2019-11-01T23:40. As it has been done in previous studies using this data [ 29 ], we defined the forecasting horizon to be 4 hours, which involves a prediction of 24 time-steps. Figure 1plots the electric demand data at different scales. As can be seen, the time series presents both weekly and daily seasonality. In general, the demand is higher at weekdays than at weekends and suffers a severe drop during night-time every day. 2.1.2. Electric Vehicles Power Consumption The second dataset gathers information about power consumption in charging stations for electric vehicles (EV) in Spain. This data was also obtained from Red Eléctrica de España and can be found at [ 30 ]. In the near future, governments will have to build infrastructures that can fulfil the demands of the increasing EV fleet. Given their limited autonomy, the prediction of EV consumption seems crucial to efficiently manage the power supply. In this article, we followed the same steps to generate the EV demand time series as in [ 31 ]. The data was collected hourly and ranges from 2015-03-02T00:00 to 2016-05-31T23:00. For each geographical area, we obtained a single value of power consumption for every hour. Later, in order to obtain a single time series, the different zones were aggregated giving the total energy consumption of the Spanish EV. The forecasting horizon was set to be 48 h, which involves a prediction of 48 time-steps. The dataset is divided into two sets: the training set contains 8759 samples corresponding to the period from 2015-03-02T00:00 to 2016-02-29T23:00; and the test set contains 2207 samples comprising the period from 2016-03-01T00:00 to 2016-05-31T23:00. As can be seen in Figure 2, the time series presents weekly and daily patterns. Since electric vehicles are most commonly charged at night, the time series presents peak values of demand during Appl. Sci. 2020,10, 2322 4 of 17 the first six hours of each day. Only Sundays and Mondays present a slightly different pattern, as it is displayed in Figure 2c. 2014-01-01 2015-01-01 2016-01-01 2017-01-01 2018-01-01 2019-01-01 2019-11-02 20 25 30 35 40 GW (a) Complete time series showing the evolution of the electric demand in Spain from 2014 to 2019. January Feb. March April May June July August Sep. October Nov. December 20 25 30 35 40 GW (b) Evolution of the electric demand in Spain during 2016. Mon 00:00 Tue 00:00 Wed 00:00 Thu 00:00 Fri 00:00 Sat 00:00 Sun 00:00 Sun 23:50 20.0 22.5 25.0 27.5 30.0 32.5 35.0 37.5 GW 1 -7 Jan 9 -15 Apr 16 -22 Jul 22 -28 Oct (c) Evolution of the electric demand during four different weeks of 2018. 00:00 02:00 04:00 06:00 08:00 10:00 12:00 14:00 16:00 18:00 20:00 22:00 24:00 22.5 25.0 27.5 30.0 32.5 35.0 37.5 GW Mon Tue Wed Thu Fri Sat Sun (d) Evolution of the electric demand within each day of the first week of February 2019. Figure 1. Line plots illustrating the electric demand time series data at different scales. Appl. Sci. 2020,10, 2322 5 of 17 2015-03-02 2016-01-01 2016-05-31 0 25 50 75 100 125 150 175 kW ( a ) Complete time series showing the evolution of electric vehicle (EV) power consumption from March 2015 to end of May 2016. Mon 00:00 Tue 00:00 Wed 00:00 Thu 00:00 Fri 00:00 Sat 00:00 Sun 00:00 Sun 23:50 0 25 50 75 100 125 150 175 kW 2 -8 Mar 2015 7 -13 Sep 2015 7 -13 Mar 2016 2 -8 May 2016 (b) Evolution of the EV power consumption during four different weeks. 00:00 02:00 04:00 06:00 08:00 10:00 12:00 14:00 16:00 18:00 20:00 22:00 24:00 0 20 40 60 80 100 120 140 kW Mon Tue Wed Thu Fri Sat Sun (c) Evolution of the EV power consumption within each day of the first week of February 2016. Figure 2. Line plots illustrating the electric vehicle power consumption data at different scales. 2.2. Methodology In this subsection, we describe the required data preprocessing steps and the fundamental concepts behind temporal convolutional networks. 2.2.1. Data Preprocessing In order to train a deep learning model that can predict several time-steps, a preprocessing stage is needed to transform the original time series data. First, we perform min-max normalisation to the entire sequence to scale the values between 0 and 1, which helps to improve the convergence of deep networks. Secondly, we transform the sequence into instances that can be used to feed the Appl. Sci. 2020,10, 2322 6 of 17 network. There exist several strategies to deal with multistep forecasting problems [ 32 ]: the recursive strategy, which performs one-step predictions and feeds the result as the last input for the next prediction; the direct strategy, which builds one model for each time step; and the multi-output approach, which outputs the complete forecasting horizon vector using just one model. As suggested in recent forecasting studies that use neural networks [ 33 , 34 ], in this work, we adopt the MIMO strategy (Multi-Input Multi-Output) which belongs to the last category. Instead of forecasting each time-step independently, the MIMO approach can model the dependencies between the predicted values since it outputs the complete forecasting window. Furthermore, this strategy avoids the accumulated errors over predictions that appear in the recursive strategy. Following this approach, a moving window scheme is used to create the input–output pairs that will be fed to the neural network. All deep learning models used in this study accept a fixed-length window as input and have an output dense layer with as many neurons as the forecasting horizon defined for each problem (24 for electricity demand and 48 for electric vehicle demand). Figure 3 illustrates the process of applying the moving window over the complete time series. As can be seen, the window slides and obtains an input–output instance at each position. While the output window size is defined by the problem, the input window size has to be decided. The optimal value can be different depending on the data, the designed model, and the forecasting horizon. In our study, we have experimented with three different sizes for the input window of each problem. The values have been carefully selected, considering the characteristics and seasonality of the datasets. For the electricity demand, we evaluate using 144, 168, and 268 time-steps as input window (which corresponds to 24, 28, and 48 h, respectively). For the power demand of electric vehicles, we consider 168, 336, and 672 time-steps as input window (which corresponds to 7, 14, and 28 days, respectively). Figure 3. Moving window procedure that obtains the input–output instances. In this example, the input and output windows have lengths of 7 and 3, respectively. Appl. Sci. 2020,10, 2322 7 of 17 2.2.2. Temporal Convolutional Neural Network TCNs are a type of convolutional neural network with a specific design that makes them suitable for handling time series. TCNs satisfy two main principles: the network’s output has the same length as the input sequence (similarly to LSTM networks); and they prevent leakage of information from future to the past by using causal convolutions [ 24 ]. Causal convolution differs from standard convolution in the fact that the convolutional operation performed to obtain the output at time t does not take future values as inputs. This implies that, using a kernel size k , the output Ot is obtained using the values of Xt−(k−1) , Xt−(k−2) , . . . , Xt−1 , Xt (Figure 4). Zero-padding of length k− 1 is used at every layer to maintain the same length as the input sequence. ( a ) Standard convolution block with two layers with kernel size 3. ( b ) Causal convolution block with two layers with kernel size 3. ( c ) Dilated causal convolution block with two layers with kernel size 2, dilation rate 2. Figure 4. Differences between ( a ) standard convolutional network, ( b ) causal convolutional network, and (c) dilated causal convolutional network. Furthermore, with the aim of capturing longer-term patterns, TCNs use one-dimensional dilated convolutions. This convolution increases the receptive field of the network without using pooling operations, hence there is no loss of resolution [ 35 ]. Dilation consists of skipping d values between the inputs of the convolutional operation, as can be seen in Figure 4c. The complete dilated causal convolution operation over consecutive layers can be formulated as follows [36]: xt l=g K−1 ∑ k=0 wk lx(t−(k×d)) (l−1)+bl!, (1) where xt l is the output of the neuron at position (t) in the l -th layer; K is the width of the convolutional kernel; wk l stands for the weight of position (k) ; d is the dilation factor of the convolution; and bl is the bias term. Rectified Linear Units (ReLU) layers are used as activation function ( g(x) = max( 0, x) ) [ 37 ]. Another common approach to further increase the network’s receptive field is to concatenate several TCN blocks, as can be seen in Figure 5[ 38 ]. However, this leads to deeper architectures with many more parameters which complicates the learning procedure. For this reason, a residual connection is added to the output of each TCN block. Residual connections were proposed by [ 39 ] in order to improve performance in very deep architectures, and consist of adding the input of a TCN block to its output (o=g(x+F(x))). All these characteristics make TCNs a very suitable deep learning architecture for complex time series problems. The main advantage of TCNs is that, similarly to RNNs, they can handle variable-length inputs by sliding the one-dimensional causal convolutional kernel. Furthermore, TCNs are more memory efficient than recurrent networks due to the shared convolution architecture which allows them to process long sequences in parallel. In RNNs, the input sequences are processed sequentially, which results in higher computation time. Moreover, TCNs are trained with the standard backpropagation algorithm, hence avoiding the gradient problems of the backpropagation-through-time algorithm (BPTT) used in RNN [40]. Appl. Sci. 2020,10, 2322 8 of 17 Figure 5. Temporal Convolutional Networks (TCN) model with 3 stacked blocks. Each block has 3 convolutional layers with kernel size 2 and dilations [1, 2, 4]. 2.3. Experimental Study In this subsection, we present the design of the experimental study carried out over the two energy-related datasets. Furthermore, we also describe the details of the parameter search process for each model architecture. 2.3.1. Models The aim of this work is to build a TCN-based deep learning model to improve the performance in energy demand forecasting tasks, in terms of both accuracy and efficiency. In order to compare the effectiveness of TCNs for this problem, we also evaluate the performance of recurrent LSTM networks—that have so far been considered the state-of-the-art in forecasting. However, given the high complexity of deep learning models, finding optimal values for the hyperparameters of these networks is a very challenging task. Therefore, we have conducted an extensive experimental study that involves more than 1900 combinations of parameters that build different convolutional and recurrent architectures. An important hyperparameter that is common to both types of architectures is the size of the input window. The possible values for past history were defined in Section 2.1 and depend on the data characteristics and seasonality. Additionally, we have searched for the best values of several parameters that are specific for TCN or LSTM architectures. In the case of TCNs, we have experimented with a different number of filters and stacked residual blocks, kernel sizes, and dilations factors. In the case of LSTMs, we have experimented with a different number of stacked layers and units. Furthermore, we have also studied the effect of training parameters in the performance of all models, such as the batch size and the number of epochs. The Adam optimiser has been selected for training the models, which has an adaptive learning rate that can improve the convergence speed of deep networks [ 41 ]. The mean absolute error (MAE) has been used as the loss function for all experiments. Table 1displays all TCN architecture configurations that have been tested over both datasets. The parameter search process has been designed considering the receptive field of neurons inside the network, that can be calculated as follows: (receptive f ield =no . stacked blocks ×kernel size × last dilation f actor) . Depending on the length of the past history window, we carefully select possible values for kernel size, stacked blocks, and dilations so that the receptive field covers the whole input sequence. For instance, if the number of stacked block increases, less dilated convolutional layers are needed, as can be seen in Table 1a,b. All these architectures are then tested with all combinations of parameters displayed in Table 1c that are common for both datasets (number of convolutional filters, epochs, and batch size). Overall, 756 (28 models from Figure 7a × 3 numbers of filters × 3 batch size × Appl. Sci. 2020,10, 2322 9 of 17 3 number of epochs from Table 1c) experiments with different configurations using TCNs have been conducted for the electric demand dataset, and 513 (12 from Figure 7b × 3 × 3 × 3 from Table 1c) for the EV power consumption data. Table 1. Architecture configuration of all TCN models for each dataset and common parameter search. a TCN architectures depending on the past history for electric demand data. Past History TCN Model Architecture Kernel Size No. Blocks Dilations 144 2 3 [1, 3, 6, 12, 24] 3 1 [1, 3, 6, 12, 24, 48] 2 [1, 3, 6, 12, 24] 3 [1, 2, 4, 8, 16] 4 [1, 3, 6, 12] 43 [1, 3, 6, 12] 4 [1, 3, 9] 6 1 [1, 3, 6, 12, 24] 2 [1, 3, 6, 12] 3 [1, 2, 4, 8] 4 [1, 3, 6] 168 2 3 [1, 5, 7, 14, 28] 3 1 [1, 5, 7, 14, 28, 56] 2 [1, 5, 7, 14, 28] 4 [1, 5, 7, 14] 4 3 [1, 5, 7, 14] 6 1 [1, 5, 7, 14, 28] 2 [1, 5, 7, 14] 4 [1, 5, 7] 288 2 3 [1, 3, 6, 12, 24, 48] 3 2 [1, 3, 6, 12, 24, 48] 3 [1, 2, 4, 8, 16, 32] 4 [1, 3, 6, 12, 24] 4 3 [1, 3, 6, 12, 24] 6 1 [1, 3, 6, 12, 24, 48] 2 [1, 3, 6, 12, 24] 3 [1, 2, 4, 8, 16] 4 [1, 3, 6, 12] b TCN architectures depending on the past history for EV power consumption data. Past History TCN Model Architecture Kernel Size No. Blocks Dilations 168 2 3 [1, 5, 7, 14, 28] 3 1 [1, 5, 7, 14, 28, 56] 2 [1, 5, 7, 14, 28] 3 [1, 5, 7, 14] 4 3 [1, 5, 7, 14] 6 1 [1, 5, 7, 14, 28] 2 [1, 5, 7, 14] 4 [1, 5, 7] 336 2 3 [1, 5, 7, 14, 28, 56] 32 [1, 5, 7, 14, 28, 56] 4 [1, 5, 7, 14, 28] 4 3 [1, 5, 7, 14, 28] 6 1 [1, 5, 7, 14, 28, 56] 2 [1, 5, 7, 14, 28] 4 [1, 5, 7, 14] 672 3 4 [1, 5, 7, 14, 28, 56] 4 3 [1, 5, 7, 14, 28, 56] 62 [1, 5, 7, 14, 28, 56] 4 [1, 5, 7, 14, 28] cParameter search for TCN models. TCN Parameters No. of filters {32, 64, 128} Batch size {64, 128, 255} No. of epochs {25, 50, 100} Table 2presents the parameters that have been studied for LSTM networks. Given the sequential processing nature of these networks, they can effectively cover the complete input sequence and capture long-term dependencies. Therefore, the parameter search, in this case, is based on trying different combinations of LSTM units that can process sequences and feed the output to subsequent stacked layers. We also consider the same possible values as above for the input window length. A total of 243 (3 past history × 3 × 3 × 3 from Table 2) experiments with different LSTM models have been carried out for each dataset. Appl. Sci. 2020,10, 2322 16 of 17 6. Fan, C.; Xiao, F.; Wang, S. Development of prediction models for next-day building energy consumption and peak power demand using data mining techniques. Appl. Energy 2014,127, 1–10. [CrossRef] 7. Torres, J.; Troncoso, A.; Koprinska, I.; Wang, Z.; Martínez-Álvarez, F. Deep Learning for Big Data Time Series Forecasting Applied to Solar Power. In Proceedings of the 13th International Conference on Soft Computing Models in Industrial and Environmental Applications, San Sebastian, Spain, 6–8 June 2018; pp. 123–133. [CrossRef] 8. Ray, P.; Mishra, D.; Lenka, R. Short term load forecasting by artificial neural network. In Proceedings of the 2016 International Conference on Next Generation Intelligent Systems (ICNGIS), Kottayam, India, 1–3 September 2016. [CrossRef] 9. Schäfer, A.M.; Zimmermann, H.G. Recurrent Neural Networks Are Universal Approximators. In Proceedings of the 16th International Conference on Artificial Neural Networks ICANN’06, Athens, Greece, 10–14 September 2006; pp. 632–640. [CrossRef] 10. Wang, Y.; Liu, M.; Bao, Z.; Zhang, S. Short-Term Load Forecasting with Multi-Source Data Using Gated Recurrent Unit Neural Networks. Energies 2018,11, 1138. [CrossRef] 11. Zheng, J.; Xu, C.; Zhang, Z.; Li, X. Electric load forecasting in smart grids using Long-Short-Term-Memory based Recurrent Neural Network. In Proceedings of the 2017 51st Annual Conference on Information Sciences and Systems (CISS), Baltimore, MD, USA, 22–24 March 2017; pp. 1–6. [CrossRef] 12. Hochreiter, S.; Schmidhuber, J. Long Short-term Memory. Neural Comput. 1997,9, 1735–1780. [CrossRef] 13. Ma, X.; Tao, Z.; Wang, Y.; Yu, H.; Wang, Y. Long short-term memory neural network for traffic speed prediction using remote microwave sensor data. Transp. Res. Part C Emerg. Technol. 2015 ,54, 187–197. [CrossRef] 14. Gensler, A.; Henze, J.; Sick, B.; Raabe, N. Deep Learning for solar power forecasting—An approach using AutoEncoder and LSTM Neural Networks. In Proceedings of the 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Budapest, Hungary, 9–12 October 2016; pp. 2858–2865. [CrossRef] 15. Fischer, T.; Krauss, C. Deep learning with long short-term memory networks for financial market predictions. Eur. J. Oper. Res. 2018,270, 654–669. [CrossRef] 16. Peng, L.; Liu, S.; Liu, R.; Wang, L. Effective long short-term memory with differential evolution algorithm for electricity price prediction. Energy 2018,162, 1301–1314. [CrossRef] 17. Amarasinghe, K.; Marino, D.; Manic, M. Deep neural networks for energy load forecasting. In Proceedings of the 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE), Edinburgh, UK, 19–21 June 2017; pp. 1483–1488. [CrossRef] 18. Almalaq, A.; Edwards, G. A review of deep learning methods applied on load forecasting. In Proceedings of the 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), Cancun, Mexico, 18–21 December 2017; pp. 511–516. [CrossRef] 19. Kuo, P.H.; Huang, C.J. A High Precision Artificial Neural Networks Model for Short-Term Energy Load Forecasting. Energies 2018,11, 213. [CrossRef] 20. Koprinska, I.; Wu, D.; Wang, Z. Convolutional Neural Networks for Energy Time Series Forecasting. In Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil, 8–13 July 2018; pp. 1–8. [CrossRef] 21. Cirstea, R.G.; Micu, D.V.; Muresan, G.M.; Guo, C.; Yang, B. Correlated Time Series Forecasting Using Multi-Task Deep Neural Networks. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management; ACM: New York, NY, USA, 2018; pp. 1527–1530. [CrossRef] 22. Tian, C.; Ma, J.; Zhang, C.; Zhan, P. A Deep Neural Network Model for Short-Term Load Forecast Based on Long Short-Term Memory Network and Convolutional Neural Network. Energies 2018 ,11, 3493. [CrossRef] 23. Shen, Z.; Zhang, Y.; Lu, J.; Xu, J.; Xiao, G. A novel time series forecasting model with deep learning. Neurocomputing 2019. [CrossRef] 24. Bai, S.; Kolter, J.Z.; Koltun, V. An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. arXiv 2018, arXiv:1803.01271. 25. Borovykh, A.; Bohte, S.; Oosterlee, C. Dilated convolutional neural networks for time series forecasting. J. Comput. Financ. 2019,22, 73–101. [CrossRef] 26. Chen, Y.; Kang, Y.; Chen, Y.; Wang, Z. Probabilistic Forecasting with Temporal Convolutional Neural Network. Neurocomputing 2020. [CrossRef] Appl. Sci. 2020,10, 2322 17 of 17 27. Wan, R.; Mei, S.; Wang, J.; Liu, M.; Yang, F. Multivariate temporal convolutional network: A deep neural networks approach for multivariate time series forecasting. Electronics 2019,8, 876. [CrossRef] 28. Spanish Public Grid (Red eléctrica de España). Available online: https://www.ree.es/es/datos/ (accessed on 25 February 2020). 29. Galicia, A.; Torres, J.F.; Martínez-Álvarez, F.; Troncoso, A. Scalable Forecasting Techniques Applied to Big Electricity Time Series. In Advances in Computational Intelligence; Rojas, I., Joya, G., Catala, A., Eds.; Springer International Publishing: Cham, Switzerland, 2017; pp. 165–175. 30. Cecovel: Centro de control de vehículo eléctrico (Monitoring Centre of Electric Vehicles). Available online: https://www.ree.es/es/red21/vehiculo-electrico/cecovel (accessed on 25 February 2020). 31. Gómez-Quiles, C.; Asencio-Cortés, G.; Gastalver-Rubio, A.; Martínez-Álvarez, F.; Troncoso, A.; Manresa, J.; Riquelme, J.C.; Riquelme-Santos, J.M. A Novel Ensemble Method for Electric Vehicle Power Consumption Forecasting: Application to the Spanish System. IEEE Access 2019,7, 120840–120856. [CrossRef] 32. Taieb, S.B.; Bontempi, G.; Atiya, A.F.; Sorjamaa, A. A review and comparison of strategies for multi-step ahead time series forecasting based on the NN5 forecasting competition. Expert Syst. Appl. 2012 ,39, 7067–7083. [CrossRef] 33. Bandara, K.; Bergmeir, C.; Smyl, S. Forecasting across time series databases using recurrent neural networks on groups of similar series: A clustering approach. Expert Syst. Appl. 2020,140, 112896. [CrossRef] 34. Petersen, N.C.; Rodrigues, F.; Pereira, F.C. Multi-output bus travel time prediction with convolutional LSTM neural network. Expert Syst. Appl. 2019,120, 426–435. [CrossRef] 35. Yu, F.; Koltun, V. Multi-Scale Context Aggregation by Dilated Convolutions. In Proceedings of the 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, 2–4 May 2016. 36. Lara-Benítez, P.; Carranza-García, M.; García-Gutiérrez, J.; Riquelme, J. Asynchronous dual-pipeline deep learning framework for online data stream classification. Integr. Comput.-Aided Eng. 2020, 1–19. [CrossRef] 37. Nair, V.; Hinton, G. Rectified linear units improve Restricted Boltzmann machines. In Proceedings of the ICML 2010—27th International Conference on Machine Learning, Haifa, Israel, 21–24 June 2010; pp. 807–814. 38. Van den Oord, A.; Dieleman, S.; Zen, H.; Simonyan, K.; Vinyals, O.; Graves, A.; Kalchbrenner, N.; Senior, A.W.; Kavukcuoglu, K. WaveNet: A Generative Model for Raw Audio. arXiv 2016, arXiv:1609.03499. 39. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [CrossRef] 40. Pascanu, R.; Mikolov, T.; Bengio, Y. On the difficulty of training Recurrent Neural Networks. arXiv 2012 , arXiv:cs.LG/1211.5063. 41. Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv 2014, arXiv:cs.LG/1412.6980. 42. Lara-Benítez, P.; Carranza-García, M. Electric Demand Forecasting with Temporal Convolutional Networks. Available online: https://github.com/pedrolarben/ElectricDemandForecasting-DL (accessed on 1 March 2020). 43. Keskar, N.S.; Mudigere, D.; Nocedal, J.; Smelyanskiy, M.; Tang, P.T.P. On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima. arXiv 2016, arXiv:1609.04836. c 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).