A comparison of time series lags and non-lags in Spanish electricity price forecasting using data science models

A comparison of time series lags and non-lags in Spanish electricity price forecasting using data science models BELÉN VEGA-MÁRQUEZ∗, Department of Computer Languages and Systems, University of Sevilla, Sevilla, Spain. JAVIER SOLÍS-GARCÍA∗∗, Department of Computer Languages and Systems, University of Sevilla, Sevilla, Spain. ISABEL A. NEPOMUCENO-CHAMORRO†, Department of Computer Languages and Systems, University of Sevilla, Sevilla, Spain. CRISTINA RUBIO-ESCUDERO††, Department of Computer Languages and Systems, University of Sevilla, Sevilla, Spain. Abstract Electricity is an indicator that shows the progress of a civilization; it is a product that has greatly changed the way we think about the world. Electricity price forecasting became a fundamental task in all countries due to the deregulation of the electricity market in the 1990s. This work examines the effectiveness of using multiple variables for price prediction given the large number of factors that could influence the price of the electricity market. The tests were carried out over four periods using data from Spain and deep learning models. Two different attribute selection methods based on Pearson’s correlation coefficient have been used to improve the efficiency of the training process. The variables used as input to the different prediction models were chosen, considering those most commonly used previously in the literature. This study attempts to test whether using time series lags improves the non-use of lags. The results obtained have shown that lags improve the results compared to a previous work in which no lags were used. Keywords:time lags, deep learning, day-ahead electricity price forecasting, time series forecasting 1 Introduction The deregulation of the electricity market worldwide in the late 1990s caused a significant change in how the market was seen. This significant change implied the start of a very competitive market in which prices were constantly changing [11]. This change challenged all players in the market, the two customers and makers, as both tried to expand their benefits and enhance their services, respectively. ∗E-mail: b[email protected] ∗∗E-mail: [email protected] †E-mail: [email protected] ††E-mail: [email protected] Vol. 32, No. 6, © The Author(s) 2024. Published by Oxford University Press. This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact jour[email protected] Advance Access published 22 March 2024 https://doi.org/10.1093/jigpal/jzae034 Downloaded from https://academic.oup.com/jigpal/article/32/6/1036/7632105 by guest on 16 January 2025 A Comparison of Time Series Lags and Non-Lags in Spanish Electricity Price Forecasting 1037 These interests have made the price one of the target variables inside this market, with specific interest paid to predict the day-ahead electricity price. This prediction of the day is a benefit for all market participants since, on the one hand, having a reliable prediction of it, electricity producers can decisively make their offers before closing time and, on the other hand, the operator of the system can control that there are no anomalies in the supply and demand of this valuable commodity [25]. Unlike different markets, the main feature of the electricity market is that it works with a commodity that cannot be stored, implying a constant attempt to maintain regularity between consumption and production [28]. Furthermore, very recently, with the growth of renewable energies [26], the price of electricity has become a volatile commodity strongly influenced by seasonality. All these characteristics make forecasting electricity prices a difficult task, one in which many exceptional components should be considered. In recent decades, several deep learning forecasting models have been proposed to solve the abovementioned challenge in electricity price forecasting. In [10] some techniques are developed: a hybrid model composed of a variational mode decomposition (VMD), a convolutional neural network (CNN) and a gated recurrent unit (GRU) for short-term electricity prediction. Zhang et al. [28] carried out a new adaptive hybrid model based on VMD, self-adaptive particle swarm optimization (SAPSO), seasonal autoregressive integrated moving average (SARIMA) and deep belief network (DBN) for short-term electricity price forecasting using data from Australia, Pennsylvania, and Spain. In [30], an optimized heterogeneous LSTM structure was proposed. It was constructed from the decomposition of the electricity price combined with a sequence model-based optimization (SMBO) to optimize its hyperparameters. A hybrid framework based on deep learning is proposed by [27] to forecast the price of electricity for the day is proposed by [27] using PJM market data. Lastly, in [8], a deep convolutional neural network is combined with forest learning of label distribution to forecast the price of electricity in Singapore, proving that this approach offers better predictions than current approaches. Machine learning models have also been proven to be practical tools for predicting electricity prices, as proposed in [3], in which a comparative study using deep learning and support vector regression was performed to predict electricity prices in smart grids. These aforementioned papers make the prediction from a univariate point of view; that is, they do not use variables exogenous to the price (multivariate methodology). Several research works have used this technique, obtaining quite promising results [12,22]. From a multivariate point of view, it is possible to obtain external information directly influencing the price, which can help model the price behavior. This study aims to decode whether or not using time instants prior to the point to be predicted is an excellent alternative to predict the price of electricity in the Spanish market. In summary, the most significant contributions of this work have been the following: −In an effort to make the training process maximally efficient, two attribute selection methodologies based on Pearson’s correlation coefficient were carried out: the first one took into account all the lags up to a previous week and the other methodology took into account the time lag of a previous week. −The experiments were carried out with four different datasets of the price in the Spanish electricity market; the difference between them is the period in which they are located. −The comparison was made using 7 different models. −The results obtained have been compared with a previous work [24] in which the same variables as in this work were used, but without taking into account temporal lags. Downloaded from https://academic.oup.com/jigpal/article/32/6/1036/7632105 by guest on 16 January 2025 1038 A Comparison of Time Series Lags and Non-Lags in Spanish Electricity Price Forecasting −It has been shown that using time lags before the instant to be predicted has been shown to improve the prediction results. The rest of this paper is organized as follows. Section 2 introduces our defined methodology in detail (forecast models, experimental setup, and evaluation procedure). In Section 3, the results are explained. Finally, the conclusions are provided in Section 4. 2 Methodology This paper aims to provide a fair comparison with previous work [24] in which no previous information was used to predict the price of electricity on the Spanish market. The first step in our framework consisted of extracting the characteristics that strongly influence the target variable, i.e., the price. We used Pearson’s correlation coefficient to extract the characteristics that best determine the price of electricity. Next, we analyze several well-known algorithms, from deep learning approaches to standard machine learning methods, to obtain the best results. 2.1 Hyperparameter optimization and training Based on the review conducted in [13], where many different architectures were compared, we have decided to use the ones with the best results: Long-Short-Term Memory (LSTM), CNN, and Temporal Convolutional Networks (TCN). Moreover, we have chosen the standard multilayer perceptron (MLP) and a new recurrent model, a GRU. We have also chosen two other popular machine learning models in the literature: Regression Trees (TREE) and Random Forests (RF). All models are listed below along with a brief description. −LSTM: Introduced in 1997, they can model temporal dependencies on larger horizons without forgetting short-term patterns [9]. LSTMs are a particular type of RNNs that have a solid ability to solve problems related to nonlinear sequential data [2]. This approach is mainly based on memory cells that replace the hidden layers of conventional neurons. −GRU: They were introduced in 1997 to learn long-term dependencies and improved further in the following years [19]. This type of network can be considered as a simplification of LSTMs in which there are two main gates, a reset gate that adjusts the new data and the data in memory and an update gate that controls the memory of the network itself [29]. −CNN: They were mainly designed for computer vision tasks; however, they have had groundbreaking results in the last decade in a variety of fields related to pattern recognition; from image processing to voice recognition [1]. CNNs consists mainly of three layers: convolutional, pooling, and fully connected layers. Convolutional layers are the most important in the training process because they use the local correlation of the information in the picture to extract features. −TCN: It is a new architecture based on CNN, but more specialized in temporal series, inspired by the Wavenet autoregressive model [15], which was designed for audio generation problems. This type of network emerged to solve the limitations found with LSTM networks, being able to efficiently extract features within the time series to be studied and even being able to predict accurately data with deformations [14]. −MLP: It is the most basic type of a feedforward artificial neural network, with a variable architecture, but, in general, consists of an input layer that acts as the receiver of the data, several hidden layers, and an output layer that predicts the output [6]. It uses the backpropagation technique in the training process. Downloaded from https://academic.oup.com/jigpal/article/32/6/1036/7632105 by guest on 16 January 2025 A Comparison of Time Series Lags and Non-Lags in Spanish Electricity Price Forecasting 1039 −TREE: It consists of a particular type of decision tree, which is characterized by being quite interpretable and having reasonably high accuracy, dealing with a continuous goal variable, such as the price of electricity [20]. The popularity of this algorithm could be explained by its flexibility and interpretability of the results. −RF: It builds a large number of regression trees to solve regression tasks. These trees, which act as regression functions, are combined using bootstrap or ensemble techniques to reach a final decision [18]. We have divided our datasets into two different sets: training and validation. The train set has been used to find the best combination of hyperparameters for each model. To validate the models, we used the second subset to obtain a metric that was not influenced by the possible overfitting of the model. In both subsets, the Multi-Input Multi-Output (MIMO) technique [13] has been used to avoid error accumulation in the predictions. A grid search methodology [21] has been used to cover as many parameters as possible. All combinations of hyperparameters used and the code implemented can be found in [23]. Both the train and the validation sets are made up of windows that contain our feature values at each hour. With one window as input, the model generates another window with the forecast values every 24 hours of the electricity price. 2.2 Validation To evaluate the performance and precision of the proposed models, we used the mean absolute error (MAE), shown in Equation 1, which measures the average absolute error between the original time series and the forecasted. This metric is mainly characterized by measuring the difference between the actual value and the prediction; therefore, the best models are those whose validation metric is close to 0, indicating that the error committed is low. In addition, MAE gives us the error on the same scale as the measured data. MAE(y,o)=mean(|y−o|)=1 n n  i=1 |yi−oi|(1) 3Results and discussion This section will explain the results obtained in the experiments. First, the chosen datasets will be presented. After this, considering the correlation coefficient, the variables selected for each dataset for each feature selection method will be shown. Finally, the best results for each dataset and each model by experiment will be presented, according to the parameterization of each one of them. 3.1 Datasets In this study, we focused our attention on forecasting electricity prices. The main objective of this study is to see how the choice of lag variables can improve the effectiveness of the prediction results. For each of the chosen datasets, three months have been selected: two months have been used to set the parameters of the models and the remaining month to validate them. Periods have been selected that present unique particularity. All data used in the study have been retrieved from [17] and will be explained below. Downloaded from https://academic.oup.com/jigpal/article/32/6/1036/7632105 by guest on 16 January 2025 1040 A Comparison of Time Series Lags and Non-Lags in Spanish Electricity Price Forecasting TABLE 1. Table with information on each data set used in this investigation. The Min, Mean, Max, and Std columns refer to the electricity price for each period, which is measured in texteuro. Dataset Time period Samples Min Mean Max Std Normal 15/09/2019 - 15/12/2019 2208 10.45 48.916 74.55 9.482 Quarantine 15/03/2020 - 14/06/2020 2208 7.07 28.096 46.99 6.73 Fraud 01/10/2016 - 31/01/2017 2208 20.32 63.464 86.4 9.827 High-Prices 02/12/2021 - 02/03/2022 2184 15.51 219.847 484.91 62.371 −Normal period: To have a reference period, a stage was chosen in which no anomalies were detected, unlike the three following data sets. −Quarantine Period: The year 2020 marked a change in many daily habits due to the global pandemic suffered. One of the most notable changes was the increase in electricity consumption, as people spent more time at home due to quarantine requirements. The three months from March 15 to June 15 2020 have been chosen to represent this stage. −Fraud Period: In 2019, the National Commission for Markets and Competition (CNMC) fined Endesa and Naturgy for altering electricity prices between October 2016 and January 2017 [16], the months chosen for this study. This period will help to test how sensitive the models are to periods that may present tax irregularities. −High-Prices Period: The end of 2021 and the beginning of 2022 have meant a radical change in the electricity bill due to an exorbitant increase in electricity prices and an increase in gas prices for various social and economic reasons [4]. Information is summarized in Table 1. It shows the periods that comprise each dataset as well as the most common statistics. It can be observed that high-prices period has the highest prices due to the economic and social circumstances currently occurring in the global environment. It is also noted that the fraud period shows higher prices than the other two remaining data sets due to suspicion of economic fraud. 3.2 Feature selection Based on references [5] and [7], we focus on the variables associated with renewable energies and those used in our initial experiment [24]. The features considered have been the Percentage of CO2free generation (Percentage), Generation of eolic energy (Eolic), Generation of solar energy (Solar), Generation of hydraulic energy (Hydraulic) and Estimation of demand (Demand). To choose the correct features for each dataset, 2 experiments with different methodologies were performed. Both experiments use Pearson’s correlation coefficient of the derivative of the time series of each variable, which allows us not to focus on finding those variables that are more related to the nominal value of the price, but that have a relationship with the evolution of the price. 3.2.1 Experiment 1 The evolution of the correlation in absolute value was analyzed for each of the variables studied. It has been decided to choose all those relative maxima above the value of 0.25 correlation, as it is the minimum threshold value above which it is considered that there is some relationship. Additionally, lags 0 and 168 have been added whenever they exceed the correlation threshold value since, being at extremes, they cannot be relative maxima, despite having high Downloaded from https://academic.oup.com/jigpal/article/32/6/1036/7632105 by guest on 16 January 2025 A Comparison of Time Series Lags and Non-Lags in Spanish Electricity Price Forecasting 1041 FIGURE 1. Evolution of the correlation of each variable for each lag in the normal period. The dotted line shows the minimum threshold (0.25) chosen to decide whether the lag was significant or not. TABLE 2. Table showing the lags selected for each dataset. It can be seen that no lags have been selected for the variables Percentage, Solar and Eolic. Dataset Price lags Demand lags Hydraulic lags Normal 0, 24, 48, 72, 96, 120, 144, 168 0, 24, 48, 72, 96, 120, 144, 168 0, 24, 48, 72, 96, 120, 144, 168 Quarantine 0, 24 0 Fraud 0, 24, 48, 72, 96, 120, 144, 168 0, 24, 48, 72, 96, 120, 144, 168 0, 24, 48, 72, 96, 120, 144, 168 High-prices 0, 24, 48, 72, 168 23, 71, 95, 119, 146, 167, 168 0, 24, 48, 72, 96, 120, 144, 168 correlations. Figure 1shows the evolution of the correlations over time for each of the variables studied. This feature selection leaves us with 25 columns selected for the normal period, 25 for the fraud period, 3 for the quarantine period, and 21 for the high-prices period. More details of the selection of features for each data set can be seen in Table 2. 3.2.2 Experiment 2 In addition to using the electricity price at the instant of prediction, for each exogenous variable, we have considered its value at the prediction time and the lag concerning one week before, giving a total of 10 study variables: 5 corresponding to the same time instant of the variable to be predicted and another 5 located a week before the time instant to be predicted. To minimize the complexity of the models, it was decided to choose the 2 most correlated features out of the 10 possible variables. Usually, the two most correlated variables were one at time 0 and the other a week earlier, as seen in Table 3, since the behavior of the variable was repeated weekly, which corresponds to the fact that we have detected that it is the lag that presents, on average (taking into account the different datasets and variables), the highest correlation values with the target variable. Table 3shows the correlation indices of each of the variables with the target variable. The variables most correlated in absolute value without considering the sign are shown in bold. Downloaded from https://academic.oup.com/jigpal/article/32/6/1036/7632105 by guest on 16 January 2025 1042 A Comparison of Time Series Lags and Non-Lags in Spanish Electricity Price Forecasting TABLE 3. Correlations using the Pearson’s correlation coefficient between the electricity price and these variables, 5 of them are at the same time instant as the target variable (No column), while the remaining 5 are the lags concerning a week before (168h column) of the previous variables. The most correlated variables in absolute value are shown in bold. Demand Percentage Eolic Hydraulic Solar No 168h No 168h No 168h No 168h No 168h Normal 0.701 0.669 –0.388 –0.383 –0.155 –0.088 0.733 0.643 –0.202 –0.202 Quarantine 0.459 0.436 –0.108 –0.115 –0.024 0.019 0.32 0.292 –0.249 –0.232 Fraud 0.746 0.728 0.203 0.149 –0.052 –0.004 0.793 0.707 –0.144 –0.149 High-prices 0.507 0.518 –0.191 –0.235 –0.029 0.034 0.664 0.599 –0.385 –0.354 TABLE 4. MAE multivariate baseline (texteuro) [24]. For each period the best result is highlighted. Dataset LSTM CNN TCN GRU MLP TREE RF Normal 4.087 3.976 3.984 3.96 3.932 4.351 4.432 Quarantine 3.401 3.426 3.524 3.191 3.361 3.626 3.836 Fraud 4.245 3.83 3.875 3.952 4.204 4.256 4.755 3.3 Experimental results 3.3.1 Baseline As mentioned previously, to see if the use of lag variables increases prediction accuracy, we should have some reference results that we can use as a baseline for comparison. These results were obtained in a previous work in which the two most correlated variables were used without considering their lags. We used Pearson’s coefficient to determine the two most important variables. The results can be seen in [24]. It should be noted that in this previous work, results were obtained only for the first 3 datasets since the last period occurred after that publication. Table 4shows the MAE results. The best values are obtained with models based on conventional deep learning methods, such as the CNN, the GRU, and the MLP, which are the most simple neural networks used in this study. 3.3.2 Experiment 1 Table 6 shows the best results obtained by each model in terms of MAE, where the best result of the dataset is highlighted in bold, those values that are worse than the baseline results are highlighted in italics, and those values that improve the original result are written normally. After adding the lags for each variable, the results do not show a clear improvement. It can be seen how the minimum value of each dataset is, at most, the baseline and how of the 21 initial results, only 7 have improved. Additionally, some of the results that have worsened have suffered a drastic increase in the error they committed, as is the case of the results of the MLP models. On the other hand, if we have to look for an architecture that has benefited from this experiment, it would be the LSTM. It can be seen that this type of model shows the best results in 2 of the 4 datasets, and it has improved 2 of the periods concerning the baseline. In the quarantine period, which is the only one in which it has worsened, it shows a low increase in error. 3.3.3 Experiment 2 The methodology for selecting the lags in this experiment shows an improvement compared to the baseline and the previous experiment. The new lag features have proven to be Downloaded from https://academic.oup.com/jigpal/article/32/6/1036/7632105 by guest on 16 January 2025 A Comparison of Time Series Lags and Non-Lags in Spanish Electricity Price Forecasting 1043 TABLE 5. Results of experiment 1 in terms of MAE (texteuro). The best results for each dataset are shown in bold. The values in italics show worsening respect to the baseline results. Dataset LSTM CNN TCN GRU MLP TREE RF Normal 3.96 4.138 4.716 4.001 5.619 4.13 4.632 Quarantine 3.557 3.361 3.642 4.049 3.56 3.846 4.548 Fraud 4.163 4.118 4.501 3.91 4.862 4.208 4.6 High-prices 18.62 19.361 19.361 19.529 27.805 26.394 22.416 TABLE 6. Results of experiment 2 in terms of MAE. The best results for each dataset are shown in bold. The values in italics show worsening respect to the baseline results. Dataset LSTM CNN TCN GRU MLP TREE RF Normal 4.010 4.386 4.067 4.110 4.556 4.498 4.233 Quarantine 3.050 3.039 2.830 3.010 2.767 4.300 4.259 Fraud 4.440 3.798 3.985 3.936 4.035 4.080 4.727 High-Prices 20.215 20.643 18.554 18.972 22.378 22.509 22.271 a better option for predicting the electricity price in the Spanish market, as they take advantage of temporal information to make much more accurate predictions in most experiments. Table 6 shows the results, where the best results for each data set are shown in bold, results that worsen concerning the baseline are written in italics, and those that show an improvement are written in normal. In summary, 11 of the original 21 baseline results are shown to be improved. Furthermore, baseline values have been exceeded in quarantine and fraud periods and have slightly worsened in the normal period. This experiment is the best result of the two performed in the High-Prices period. The quarantine period is the one that benefits the most from this methodology, notably improving many of its results and obtaining the best with the MLP architecture. For the other three remaining models, the best performing models were TCN for the normal period, which is the most disadvantaged in this methodology, TCN again for the high-prices period, and CNN for the fraud period. The temporal nature of the new variables has made it possible for the TCN network to take advantage of this fact, thus being the best performing network for temporal data. It is worth noting that the last row of results, corresponding to the high price period, obtains the highest error among all data sets studied. This increase may be because prices in this period were much higher, so the error committed is directly proportional. 3.3.4 Overall comparison In order to make a fairer comparison between the three methodologies, only the first three periods should be considered, since these are the three that are included in the three experiments. Figure 2shows the average MAE obtained with each of the models for the first three periods. Looking at this figure, there are several interesting aspects to discuss; as commented in previous sections, the best results have been obtained on average with experiment 2, this may be because adding more lags may be in some cases somewhat counterproductive, so it is always interesting to have a reliable mechanism for lag selection, such as the one we propose in experiment 2. Downloaded from https://academic.oup.com/jigpal/article/32/6/1036/7632105 by guest on 16 January 2025 1044 A Comparison of Time Series Lags and Non-Lags in Spanish Electricity Price Forecasting FIGURE 2. Comparison in terms of MAE between the previous proposal and the new methodologies. For each of the models used, the final average obtained is shown, taking into account each of the metrics obtained for each time period. The most significant difference in results is observed for the TCN and MLP models. Overall, these results show improvement whenever deep learning models are used. In contrast, the results are worse for algorithms that are not deep learning (TREE, RF) when lag variables are added. 4Conclusions The results of this study reveal that the temporal nature of the data is something to consider when choosing the best prediction algorithm. Exogenous variables have also been shown to be an excellent alternative for predicting time series in the Spanish electricity market. The main objective of this work has been to develop an experimental framework based on previously published work [24]. The comparison aimed at observing whether the use of new variables would lead to improvement. The same variables were taken into account as in the previous work; the difference is that their time lags were also considered, thus having the information for a previous week at the time to be predicted. Two different experiments were used to select the characteristics: the first one took into account all the lags up to a previous week and chose only those with a correlation greater than 0.25, while the second experiment only took into account, for each variable, the lag of a previous week and the same one referred to the instant to be predicted and chose the two most correlated ones. To make the comparison, three different periods were used, the same as those used previously. In addition, a new and more current period has been added, coinciding with the period of price increases in Spain. The results obtained have shown that the time instant that we try to predict significantly influences the final result. So many socioeconomic factors must be considered when trying to predict any of the variables with which we work within the Spanish electricity market. Considering one prediction metric, MAE, it has been observed that between the two proposed experiments, the first one was worse than the second one, meaning that a higher number of lags is not always better since it may be including unnecessary noise in the predictions. Therefore, it is always necessary to make a previous study of correlations and choose the most correlated lags. Moreover, it has also been observed that the biggest differences in terms of results are those obtained Downloaded from https://academic.oup.com/jigpal/article/32/6/1036/7632105 by guest on 16 January 2025