A hybrid approach for high precision prediction of gas flows

Petkovic, Milena et al. Article — Published Version A hybrid approach for high precision prediction of gas flows Energy Systems Provided in Cooperation with: Springer Nature Suggested Citation: Petkovic, Milena et al. (2021) : A hybrid approach for high precision prediction of gas flows, Energy Systems, ISSN 1868-3975, Springer, Berlin, Heidelberg, Vol. 13, Iss. 2, pp. 383-408, https://doi.org/10.1007/s12667-021-00466-4 This Version is available at: https://hdl.handle.net/10419/287034 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/ Vol.:(0123456789) Energy Systems (2022) 13:383–408 https://doi.org/10.1007/s12667-021-00466-4 1 3 ORIGINAL PAPER A hybrid approach forhigh precision prediction ofgas flows MilenaPetkovic1 · YingChen2 · InkenGamrath1· UweGotzes3· NataliaSeliniHadjidimitrou4 · JaninaZittel1 · XiaofeiXu2 · ThorstenKoch1,5 Received: 13 February 2020 / Accepted: 7 June 2021 / Published online: 12 August 2021 © The Author(s) 2021 Abstract About 23% of the German energy demand is supplied by natural gas. Additionally, for about the same amount Germany serves as a transit country. Thereby, the German network represents a central hub in the European natural gas transport network. The transport infrastructure is operated by transmissions system operators (TSOs). The number one priority of the TSOs is to ensure the security of supply. However, the TSOs have only very limited knowledge about the intentions and planned actions of the shippers (traders). Open Grid Europe (OGE), one of Germany’s largest TSO, operates a high-pressure transport network of about 12,000 km length. With the introduction of peak-load gas power stations, it is of great importance to predict in- and out-flow of the network to ensure the necessary flexibility and security of supply for the German Energy Transition (“Energiewende”). In this paper, we introduce a novel hybrid forecast method applied to gas flows at the boundary nodes of a transport network. This method employs an optimized feature selection and minimization. We use a combination of a FAR, LSTM and mathematical programming to achieve robust high-quality forecasts on real-world data for different types of network nodes. Keywords Gas forecast· Time series· Hybrid method· FAR· LSTM· Mathematical optimisation * Milena Petkovic petk[email protected] 1 Zuse Institute Berlin, Berlin, Germany 2 National University ofSingapore, Singapore, Singapore 3 Open Grid Europe GmbH, Essen, Germany 4 University ofModena andReggio Emilia, Modena, Italy 5 Technische Universtität Berlin, Berlin, Germany 384 M.Petkovic et al. 1 3 1 Introduction About 23% of the German (and European) energy demand is met by natural gas [2]. Additionally, for about the same amount Germany serves as a transit country. Thereby, the German network represents a central hub in the European natural gas transport network. In light of the German Energy transition (“Energiewende”) with an increasing share of renewable energy sources as well as the envisioned international transition towards substantially less fossil fuels and related greenhouse gas emissions, the importance of natural gas will increase even more. A critical task of gas power plants is to deliver electricity in peak load situations when electricity from renewable energy sources is not sufficient to cope with the demands. From the gas network point of view, this leads to huge gas demands on very short notice. The gas transmission network is run by transmission system operator, hereinafter referred to as TSOs. An integrated organization would conduct jointly operations including gas trading, running storage facilities and operating the transmission network. Thereby, the network capacities could affect transport requirements. However, TSOs face novel challenges to ensure the security of supply caused by the liberalization of the European gas markets [1], which makes TSOs no longer allowed to own, trade or store gas. Instead, trading will be conducted by independent companies to ensure discriminatory-free access to the transport network for all traders. Therefore, natural gas forecasting has become a fundamental input to the TSOs’ decision-making mechanisms. Meanwhile, the natural gas market is becoming more and more competitive and is moving towards more short-term planning, e.g., day-ahead contracts, which makes the dispatch of natural gas in the pipeline network even more challenging [12]. Therefore, a high-accuracy and high-frequency forecasting of local supplies and demands of natural gas consumption is essential for efficient network operation of TSOs. Although on the contractual level all gas transports of a market area have to be balanced, this needs only to be achieved on average over time. Some outflow might actually only be balanced by an inflow at a later time. Despite these challenges for the TSOs they need to meet all transport demands. The TSO has the obligation to monitor the situation, foresee possible shortages and react accordingly to ensure the safety of supply. Since changes in gas networks happen rather slowly it is therefore extremely important to have accurate forecasts on the demands and supply of the network to be able to react on time. We collaborated with one of the biggest German TSOs, operating a gas network with a pipe length of about 12,000 km in total (see Fig.1), to improve their hourly forecasts for demand and supply. We aim to: • Predict as precisely as possible the average hourly gas flows for the upcoming gas day, i.e., from 6 am to 6 am, just before the start of the gas day (at about 5:59 am); • The prediction needs to be appropriate for all different types of nodes ranging from connections to other networks or countries to industrial users or municipal consumers, leading to very diverse data characteristics; 385 1 3 A hybrid approach forhigh precision prediction ofgas flows To reach these goals, we investigated real data from the transport network operated by Open Grid Europe. We propose a powerful and robust hybrid forecast model that benefits from the combination of state of the art forecasting approaches and optimisation, leading to improved forecast accuracy. We interpreted the most important features that our model automatically selects. In the following, the next subsections present nomenclature and an overview of related work. Section2 describes the data we used in this study. Section3 gives Fig. 1 Map of the gas transmission network operated by Open Grid Europe 386 M.Petkovic et al. 1 3 details on the proposed models. Section4 describes the evaluation methodology and the evaluation of computational experiments. Finally, we draw some conclusions. 1.1 Nomenclature Open Grid Europe GmbH OGE Municipal power stations MUN Power stations and industry IND Storage STO Transfer points to other networks NET Mathematical programming MP Linear program LP Mixed integer linear programming MILP Functional autoregressive FAR Long short term memory LSTM Hybrid model HYB Heating day degree HDD Baseline forecast (persistance) BAS 1.2 Related work Models on natural gas demand forecasting are mainly focused on long term issues. There are quite some publications regarding electricity demand forecasting, (see, e.g. [3, 30, 32, 47]) but electricity behaves very differently from gas. A survey on models to predict natural gas consumption published between 1949 and 2010 is presented by [42] who evidences that only a few works are focused on hourly gas flow prediction. A more recent survey [51] considers 187 papers published between 2002 and 2017. The authors point out that the majority of works provide daily predictions and recognize that neural networks are the most used models. The authors also show that, on the considered period, most of the works were performed at an aggregated level (i.e. country or city) and only three papers proposed models to forecast the hourly gas consumption. In [49], two neural networks were tested to forecast natural gas consumption based on historical data and environmental variables. The authors found a better prediction accuracy when using the multi-layer perceptron compared to the radial basis function. In [48], a model similar to radial basis neural network was proposed to predict gas consumption in a distribution system. In this work, input variables were selected using a genetic algorithm. Residential hourly gas consumption was predicted with neural networks by [17]. In this work, the heating degree-hour method which considers the gap between outdoor and indoor temperature was considered. The best hyper-parameters configuration consisted of 29 neurons, a feed-forward backpropagation algorithm and tangent, sigmoid and linear functions for the input, 387 1 3 A hybrid approach forhigh precision prediction ofgas flows hidden and output layers respectively. Similarly, [45] proposed neural networks to forecast residential natural gas demand. The proposed network consisted of a multilayer perceptron with one hidden layer. The input features included calendar (i.e. month, day of the month, day of the week, hour) and weather (temperature) information. The authors found that the average prediction error was higher during the winter months because gas flow was higher. More recently, [23] compared several machine learning models to predict residential natural gas hourly demand and found that recurrent neural network and linear regression were the most accurate models. The prediction results of monthly gas consumption of residential buildings using Extreme Learning Machine (ELM), artificial neural networks (ANNs) and genetic programming (GP) were presented by [24]. The ELM is characterized by higher training speed compared to backpropagation and it was found to perform better, in terms of RMSE, compared to the other two techniques. In [26] the authors set up a two stages methodology to predict daily gas consumption of utility companies. In the first stage, two NNs are run in parallel to produce daily forecasts; in the second stage, a nonlinear transformation of some features of the input vector is performed. The combination of the two stages is based on several methods such as average forecast, recursive least squares, etc. The results show that the mix between the two forecasters has higher accuracy although the combination of the two models increases the complexity. Overall, these works show that the consumer profile is very important when forecasting gas flow. In this regard, [38] identified seventeen groups profiles, based on their historical consumption and predicted daily gas demand. The overall prediction was obtained from the combination of single predictions. The backpropagation algorithm optimized with a genetic algorithm was implemented by [54] to increase the training speed and to achieve a global minimum. The authors predict next day gas loads based on temperature and weather conditions. Furthermore, the authors tested the algorithm on a three years real dataset recorded in Shanghai to predict one month and a half gas load. Similarly, [55] propose a recurrent neural network to predict daily gas flow. The Output-Input-Hidden Feedback-Elman neural network takes into account, not only the hidden nodes’ feedbacks but also considers the output nodes’ feedbacks. The results improved compared to these obtained with the standard Elman network. However, the authors recognize that further research is needed to forecast gas demand during holidays. In [4], an adaptive network-based fuzzy inference system (ANFIS) consisting of a neural network integrated with fuzzy logic was proposed to forecast short term natural gas demand. The main advantage of this model was its ability to handle uncertainty, noise and non-linearity in the data and, compared to standard neural network models, provided more accurate results. Wavelet transform has been deployed by [44] to decompose the hourly gas demand time series and Bi-LSTM and LSTM are optimized using genetic algorithm. The model was applied to winter data on which it has shown good prediction accuracy. Several static and adaptive models have been tested by [37] for short-term gas consumption forecast (random-walk, temperature correlation model, linear regression model, ARX, adaptive (recursive) linear auto-regressive model (RARX), neural network (NN), Recurrent NN, Support Vector Regression). They found that the best performance was obtained by the RARX of order 3. Furthermore, they found that 388 M.Petkovic et al. 1 3 nonlinear models such as neural networks and support vector machines had a lower generalization capacity compared to linear models. Finally, they concluded that the adaptive models overall performed better than static models. The traditional approaches are regression and econometric models. In this regard, the performance of non linear mixed effects, ARIMAX and ARX models to predict gas consumption of 62 residential and small commercial customers was assessed by [10]. The authors forecast daily consumption of an entire month based on the previous 18 months. The time series included zero flows and missing data which were excluded for the training process. The prediction performance was similar in terms of daily mean absolute error which was close to zero for all the tested models. Thus, the authors propose to combine multiple models although they recognize that this might be a difficult task because of increased computational complexity. Multiple linear regression has been proposed by [40] who predicted annual gas consumption based on socio-economic variables (GDP and inflation in the case of Turkey) that have been selected based on their statistical significance. Based on the forecast, the authors propose alternative energy policies. Robust least square method combined with log-linear Cobb–Douglas model has been proposed by [15]. The authors compared the proposed robust and ordinary least square methods for the yearly forecast of the natural gas demand in Brazil, considering the total demand as well as the industrial and power sectors demand. The authors showed that using the proposed model can be very useful when a large amount of past data is not available, which is usually necessary for the calibration of more sophisticated forecast models. A hybrid model formed by a grey model and an autoregressive integrated moving average model has been proposed by [52] to predict monthly shale gas production. The authors conclude that the results of the combined model are more accurate than the single linear and nonlinear models. In [33], Multivariate Adaptive and Conic Multivariate Adaptive Regression Splines were proposed to predict residential daily gas demand. The two models provided better results in terms of prediction errors (MAE and RMSE) compared to these obtained with linear regression and neural networks. In [41], the nonlinear characteristics of the natural gas consumption is modeled with several Grey models that are compared to predict the yearly natural gas consumption in China. Nonlinear programming and genetic algorithm have been proposed by [19] to predict natural gas consumption in the residential and commercial sectors on a yearly basis. Similarly, [25] proposed the breeder hybrid algorithm which consists of three steps for natural gas flow demand forecast. In the first stage, the coefficients of a nonlinear regression model are estimated. Successively, the estimates are improved using a genetic algorithm. Finally, the optimized coefficients are deployed as initial solutions for the simulated annealing. Nearest neighbor and local regression were proposed by [6] to predict gas flow in a small gas network with a 15 minutes resolution. The authors evidence the importance of environmental variables such as the temperature. Their method allowed to detect anomalies and the consumption patterns based on one year historical data. In the literature, there are also combinations of several methods to predict one day-head natural gas consumption. In [34], the time series were decomposed into low-frequency and high-frequency components using Wavelet transform. In a second step, the genetic algorithm and Adaptive Neuro-Fuzzy 389 1 3 A hybrid approach forhigh precision prediction ofgas flows Inference System were deployed to predict each of the decomposed time series. The output was finally fed into a feed-forward neural network to refine the prediction. The research was focused on different types of natural gas distribution points. The authors obtained better prediction results using the data of distribution points located near the city center. Neural networks have been also compared to the performance of autoregressive models. In [46], for instance, short term natural gas consumption in Turkey was predicted using SARIMAX model and Neural Networks (Multilayer and Radial Basis) and multivariate regression. They found that SARIMAX had better prediction performance. The temperature correlation model, proposed by [43], was compared with several configurations of ARX, stepwise regression, Support Vector Regression and neural network. The authors found that SVR and NN performed better on the training set, while high order ARX model performed better on the test set. Support Vector Regression has been deployed with false neighbours filtered approach to predict short term natural gas consumption [56]. The local predictor was based on the nearest neighbour approach so that the Euclidean distance between the training and test data and the neighbour filter was applied to determine the validity of the predicted values based on the exponential separation rate. The authors obtained better performance prediction compared to ARIMA, neural networks and Support Vector Regression. Overall, the analyzed literature shows that there are few works that are focused on the comparison between methods to predict hourly gas flow of different types of nodes in a gas network or combining the advantages of different forecasting methods to a hybrid model for hourly gas flows. Therefore, we propose a hybrid model based on optimisation and machine learning and compare its results to four different models to predict hourly gas flow. To address the heterogeneity of the time series for the different node types we compare results obtained for four different types of nodes. 2 Data We consider high-resolution natural gas inflows and outflows in the high-pressure gas pipeline network operated by Open Grid Europe GmbH (OGE). The gas transmission network has more than 1000 boundary nodes which can be classified into four different groups: • Network Transfer Points (labeled NET) are large nodes with natural gas imported and exported to other networks mostly outside Germany. These can be entries and/or exits. • Municipal Utility nodes (MUN) serve residential and small commercial constituents and are always only exits. They are often temperature dependent, exhibit daily and seasonal patterns, and simultaneously are influenced by weekends/holidays. • Industry and Power Stations (IND) represent electricity generation and factory production nodes. These are also always exits and naturally exhibit weekly patterns due to working routines. 390 M.Petkovic et al. 1 3 • Storage nodes (STO) usually have a large number of zero flow hours with some substantial, often constant transfer in between. The nodes can always be both entries and exits. While, in principle, we know the above classification, it is proved not to be reliable regarding the behavior of the nodes, so we will not use this information as part of the forecast, but just to explain certain behavior. Table1 depicts the number of nodes belonging to the different groups and the percentage of gas flow explained by each group. As an illustration, we carefully selected three nodes for each type. The three network nodes we selected occupy 22% of the whole network flow. The municipal nodes are considered important by the TSO. The Industry nodes selected represent power plants and play a key role in energy generation with high renewable energy shares, as they are fast to start and can produce the necessary energy in times of peak demand. For the representative nodes from the Storage group, we selected the most frequently used nodes in the observed period. Figure2 shows normalized (to the range of [0, 1]) flows of the nodes considered in this study. For each node, the gas flows are measured hourly. Additionally, we were given the average daily temperatures measured at the nodes. Some statistical properties of selected nodes are given in Table2. As can be seen from Fig.2 and in Table2 some nodes have continuous flow, while other are active only occasionally. Storage nodes have the highest percentage of zero flows. For the ones considered in this study hours with zero flow amount for 26–53% of the time. Network nodes show the highest variability and are always inflows. Industry nodes are always outflows and are clearly not temperature dependent. Municipal nodes are usually temperature dependent and have strong daily, weekly and seasonal patterns. 3 Methods Many research studies showed that combining forecasts improves accuracy relative to individual forecasts [16, 27]. In this section, we will first present three different individual forecasting methods; Functional AutoRegressive (FAR), Long Short- Term Memory Network (LSTM) and Mathematical Programming (MP) model. Then we will propose a hybrid model (HYB) based on MP method which is using the output of two other forecasting models, FAR and LSTM, as additional inputs (features). Table 1 Number of nodes and percentage of flow Type # flow(%) Network 34 72.95 Municipal 726 16.68 Industry 234 4.26 Storage 14 6.11 397 1 3 A hybrid approach forhigh precision prediction ofgas flows and consequently the final LP problem becomes Furthermore, we can force our model to be unbiased by requiring and setting bounds for the weights l≤ w h,i≤u to limit the influence of a single specific feature. For each day in the test set (out of sample days that we want to forecast) the forecasted flow values are computed by first computing the weights via an LP with 16 weeks of historical data and then using the weighted sum of features (9) for each hour to forecast the flow values. The lower and upper bounds for the weights are set to l=−2 and u=2 , respectively. For the computation of the forecasted flow values, it might be that also forecasted flow values of prior hours are used as input values for calculating the features, if the corresponding hours do not lie in the past. 3.4 Training: feature selection In the training procedure of this method, a slightly different model is used which automatically chooses for each hour the B features which are most important, to limit over-fitting in the LP. Therefore, we add additional binary variables xh,i to the problem, which determine whether feature i is chosen for hour h, i.e., whether the weight of feature i and hour h is not equal to zero. Then, we link these variables to the weight variables and limit the number of chosen features by B The solution of the resulting MILP leads for each hour h to one feature set Fh of at most B features which are most important for this hour. The list of features used in this study is presented in Table 4. The whole set of features F we used consists of 29 different features based on historical flow values, one temperature feature and two different features describing position of the predicted gas day in the week and the offset feature. Using sensitivity analysis the number of chosen features was limited to six. One year of historical measurements was min ∑ d∈D,h∈H (e+ d,h+e− d,h) subject to ∑ i∈F fh,i(d)⋅wh,i−md,h=e+ d,h−e− d,hfor all ∈D,h∈H e+ d,h,e− d,h≥0 w h,i ∈ℝ ∑ d∈D,h∈H (e + d,h−e − d,h)= 0 xh,i⋅l ≤ w + h,i≤ xh,i⋅ u ∑ i∈F xh,i ≤B 398 M.Petkovic et al. 1 3 used for training and selecting optimal set of features for every node and hour. Figure3 is showing the heatmap of selected features for MP model for each group of nodes summed up for 24 h. For all nodes the feature representing the flow of the previous hour ( f1 ) is the mostly used feature. For all hours except the first predicted gas hour this feature value is calculated based on the the forecasted flow of previous hour when the final forecast is calculated. The same hour yesterday( f4 ) is also widely selected among all groups. As it was expected Weekend ( f31 ), Evening ( f32 ) as well as Mean temperature difference feature ( f30 ) are usually chosen only for the Municipal utilities since the behaviour of those nodes shows strong daily, weekly and seasonal patterns. In the case of Industry nodes the features of Mean flow of the same and previous day ( f29,f15 ) together with the Ratio features f11,f12 are the Table 4 List of features Feature Description f1(d,h)= { m(d,h−1), if h>0 m(d−1, 23) otherwise Prior hour f2(d,h)=m(d−1, 0) First hour yesterday f3(d,h)=m(d−1, 23) Last hour yesterday f4∶10(d,h)=m(d−(1, 2, …,7),h) The same hour 1,2,...,7 days ago f11∶12(d,h)=m(d−1, 0(h))∕m(d−2, 0(h)) Ratio first(same) hour yesterday first(same) hour 2 days ago f13∶14(d,h)=m(d−1, 0(h)) − m(d−2, 0(h)) Difference first(same) hour yesterday, first(same) hour 2 days ago f 15∶21 (d,h)=1∕24 �∑h∈H m(d−(1, 2, …,7),h) � Mean flow 1,2,...,7 days ago f22(d,h)=f15(d,h)∕f16(d,h) Ratio mean flow yesterday, 2 days ago f23(d,h)=f15(d,h)∕f21(d,h) Ratio mean flow yesterday, 7 days ago f24(d,h)=f15(d,h)∕ � 1∕24 �∑h∈H m(d−8, h) �� Ratio mean flow yesterday, 8 days ago f25(d,h)=f15(d,h)−f16(d,h) Difference mean flow yesterday, 2 days ago f26(d,h)=f15(d,h)−f21(d,h) Difference mean flow yesterday, 7 days ago f 27( d,h )= f 15( d,h )−( 1 ∕ 24 (∑h∈H m ( d − 8, h ))) Difference mean flow yesterday, 8 days ago f28(d,h)= { 0, if h=0 m(d,0) otherwise First hour today f29(d,h)= � 0, if h=0 1∕h ∑ h−1 i=0 (d,i) otherwise Mean flow today f30(d,h)=td−td−1 Difference mean temperature today and yesterday f31(d,h)= { 1, if day ∈{Saturday,Sunday } 0, otherwise Weekend f32(d,h)= { 1, if day ∈{Friday,Saturday } 0, otherwise Evening f33(d,h)=1 Offset 399 1 3 A hybrid approach forhigh precision prediction ofgas flows most frequently chosen features. For Transfer Points and Storages features of mean flow of the same and previous day ( f29,f15 ) are also dominating ones except for first gas hour where this pattern is not present. For all observed nodes the Offset feature (representing the bias in the model) is selected very frequently. Figure 4 shows a scatter plot of flow amount versus the three most frequently chosen features among different types of nodes for all hours of the day. It can be seen that the flow depends linearly on f4 while other features are showing a nonlinear dependency. 3.4.1 Hybrid model The main advantage of the mathematical programming method (MP) proposed in this paper is flexibility in the sense that adding new features when they are available in order to improve the forecast is very simple. In this section, we propose a hybrid model(HYB), combining mathematical programming (MP) with the two other proposed methods by adding the outputs from LSTM and FAR model as an exogenous inputs to the MP model. The optimal sets of features chosen for every node and hour were kept from previous MP training and extended with the forecasts from the LSTM and FAR model as additional features. The final forecast is calculated as weighted sum of all features from the extended features set: New optimal weights are calculated by running an LP: (10) pd,h= ∑ i∈F fh,i(d)⋅wh,i+LSTM (d,h)⋅wh, LSTM +FAR (d,h)⋅wh, FAR f1 f2 f3 f4 f5 f6 f7 f8 f9 f1 0 f1 1 f1 2 f1 3 f1 4 f1 5 f1 6 f1 7 f1 8 f1 9 f2 0 f2 1 f2 2 f2 3 f2 4 f2 5 f2 6 f2 7 f2 8 f2 9 f3 0 f3 1 f3 2 f3 3 NET MUN IND STO 0 10 20 30 40 50 60 70 Fig. 3 Heatmap of selected features for different node group 400 M.Petkovic et al. 1 3 Fig. 4 Scatter plot of some frequently chosen features vs flow 401 1 3 A hybrid approach forhigh precision prediction ofgas flows where LSTMh(d) and FARh(d) represent the forecasted values obtained from the LSTM and FAR models, respectively. 4 Testing andresults 4.1 Influence oftemperature The temperature is one of the most important factors that influence gas consumption. When the natural gas is consumed for heating such as in residential areas, the temperature usually has an inverse relationship with gas consumption which is also dependent on other environmental data such as the time of day, the day of week, the season, etc. Furthermore, temperature and time of day are the factors that mostly impact the forecast error [45]. The majority of models presented in the literature are focused on residential and small commercial consumer at individual or aggregate level. Several authors have considered the temperature (i.e. [19, 33, 34]) or meteorological data [46] in their models to forecast gas flow and reduce the prediction error. In [50], the authors pointed out that the nonlinear characteristics of temperature has been assessed long time ago and gas consumption is proportional to the Heating Degree Day (HDD) [5]. This proportionality is evidenced when plotting the average daily temperature versus the gas consumption. The scatter plots of daily changes of temperature versus daily changes of gas flow of the nodes considered in this work are shown in Fig.5. Storage nodes have a high percentage of zero flows, those are the nodes that better approximate the non linear relationship expressed by the HDD. As expected, the Municipal nodes present a positive correlation between the flow and the temperature. Among the network nodes, one of them presents a negative relationship with the temperature. The remaining nodes appear to be independent from the temperature. 4.2 Results 4.2.1 Objectives, setup andevaluation metrics In this paper, we observed a data set of hourly gas flow time series from 12 nodes with 17.520 observations (2 years). min ∑ d∈D,h∈H (e+ d,h+e− d,h) subject to pd,h−md,h=e+ d,h−e− d,hfor all d∈D,h∈H pd,h=∑ i∈F fh,i(d)⋅wh,i+LSTMh(d)⋅wh, LSTM +FARh(d)⋅wh , FAR e+ d,h,e− d,h≥0 w h,i ,w h, LSTM ,w h, FAR ∈ℝ 402 M.Petkovic et al. 1 3 Our goal is to predict values pd,0 to pd,23 for a given d∈D . Of course only data for days d−1 and earlier can be used. All proposed methods are tested on the last 60 days of the data set. Our basis comparison are the mean absolute deviation (MAD) between the hourly forecast and the measured flow during one day (24 h) ahead forecast, defined as Fig. 5 Scatter plot of gas consumption daily change vs temperature change 403 1 3 A hybrid approach forhigh precision prediction ofgas flows and mean absolute percentage error (MAPE) defined as All results are also compared to a baseline (BAS) forecast defined as Even though our main goal is to predict, as precisely as possible, the average hourly gas flows for the next 24 h, we adopted mean daily errors (MAD and MAPE) as accuracy metrics having in mind that the transport system operator has certain flexibility throughout the day. In particular, this means that, in principle, the maximum hourly error could be compensated by technical measures of TSO as long as it is not prolonged for several hours and have same direction. 4.2.2 Forecasting results Mean MAD and MAPE (over 60 days in the test set) achieved by proposed models are presented in the Table5. It can be seen that between individual forecasting methods the LSTM model is the most robust one and obtained the best results for nodes from all 4 behavior groups. The MP model achieved the best results for all Municipal utilities, and the performance is especially good for MUN2 node where all models had a particularly high MAPE. FAR model outperformed others for the Industry node IND2. Even (11) MADd∶= 1∕h ∑ h| ph−mh | (12) MAPE d∶= 1∕h ∑ h| (ph−mh)∕mh | (13) ph,d∶= mh,d−1 Table 5 Comparison of mean daily performance The best performance is marked in bold MAPE MAD MP FAR LSTM HYB BAS MP FAR LSTM HYB BAS NET1 0.128 0.131 0.035 0.033 0.133 1269 1350 357 331 1366 NET2 0.176 0.195 0.075 0.088 0.211 727 828 328 378 901 NET3 0.084 0.096 0.056 0.055 0.098 1855 2245 1439 1393 2168 MUN1 0.032 0.101 0.09 0.030 0.069 6.36 19.03 29.02 5.97 11.47 MUN2 0.327 0.370 0.588 0.324 0.355 65.67 72.90 101.18 64.8 71.1 MUN3 0.064 0.088 0.071 0.064 0.089 5.36 7.37 5.16 5.61 7.41 IND1 0.131 0.137 0.109 0.114 0.171 19.37 19.99 15.64 16.66 25.5 IND2 0.035 0.028 0.057 0.030 0.034 3.85 3.19 6.24 3.71 3.76 IND3 0.103 0.135 0.087 0.086 0.142 9.78 12.32 7.48 7.45 13.44 STO1 0.550 0.689 0.313 0.291 0.764 609.28 753.42 313.79 294.09 681.31 STO2 0.298 0.343 0.226 0.205 0.411 428.24 560.78 384.33 329.54 624.38 STO3 0.895 0.970 0.475 0.502 0.997 895 1155 909 816 1442 404 M.Petkovic et al. 1 3 though FAR errors are slightly higher than other two proposed models (MP and LSTM) it can be seen that for most of the nodes the performance is very similar. For Storage nodes with intermittent behaviour none of the proposed individual methods has demonstrated adequate accuracy. The Hybrid model showed an improvement for all 4 groups nodes. The improvement is especially significant for Storage nodes, where the average MAPE is lower for more than 2(%) for nodes STO1 and STO2. For node STO3 the LSTM model has the lowest MAPE but the lowest MAD is achieved by the HYB model. The four types of nodes considered in the paper have very different behaviors, and the time series of the corresponding flows can be forecasted with wide accuracy ranges. The Storage nodes have a characteristic behavior with the intermittent flow of a large scale which, in addition, can have both directions. The MUN2 node also shows some intermittent behavior, especially in the test period, which results in higher mean daily errors. Figure 6 shows calculated 24 h ahead forecast and the measured flow of all proposed models for a 1 week period. The proposed Hybrid model is built on the top of Mathematical Programming (MP) method, which represents a weighted sum of features (calculated on previous flow values as well as some exogenous variables) chosen by Mixed Integer Programming (MIP) in the offline regime. The Hybrid model’s main advantage is its property to easily include new features like forecasts from different models in order to improve the final result. Using the Hybrid model, we are ensuring that the provided forecast is at least ‘as good as’ the best model’s result, always taking advantage of discarding the influence of previously inaccurate model. Even though for some nodes, single forecast models outperform the Hybrid model, the proposed method shows robust, stable accuracy very similar to the best individual model and, in some cases, brings a significant improvements. For example, in the case for MUN2 node, which shows some intermittent behaviour and significant changes in the daily levels, the LSTM model fails to give a good forecast with the mean daily MAPE of more than 50%. Simultaneously, the Hybrid model successfully sets the weights for the corresponding features and discards LSTM forecast, using the linear combination of other features to provide the forecast, which reduces the MAPE by 26%. 5 Conclusions In this paper, we proposed a robust and powerful hybrid forecast model combining Mathematical Programming with Functional AutoRegressive and Long Short Term Neural Network model for forecasting gas flows at the boundary nodes of gas transport network. Our experiments are based on real world data from one of Germany’s largest transmission system operators, Open Grid Europe. We showed that the proposed method is appropriate for choosing optimal set of features and forecasting various behaviours from different nodes groups in the complex gas transmission network. From obtained results it is clear that even though in some specific cases single forecast models outperform the Hybrid model, the proposed 405 1 3 A hybrid approach forhigh precision prediction ofgas flows method can achieve stable accuracy close to the best individual model and in some cases brings a significant improvement to the forecast quality. Acknowledgements This work has been supported by the Research Campus Modal (Mathematical Optimization and Data Analysis Laboratories) funded by the German Federal Ministry of Education and Research (fund numbers 05M14ZAM, 05M20ZBM). Fig. 6 24 hours ahead forecast for 1 week test set 406 M.Petkovic et al. 1 3 Funding Open Access funding enabled and organized by Projekt DEAL. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. References 1. Regulation (EC) no 715/2009 of the European Parliament and of the Council on conditions for access to the natural gas transmission networks (2009) 2. The Arbeitsgemeinschaft Energiebilanzen (AG Energiebilanzen): Evaluation tables of the energy balance for Germany-Energy data for the years 1991–2018 (2019) 3. Alfares, H., Nazeeruddin, M.: Electric load forecasting: literature survey and classification of methods. Int. J. Syst. Sci. 33(1), 23–34 (2002). https:// doi. org/ 10. 1080/ 00207 72011 00674 21 4. Azadeh, A., Asadzadeh, S., Ghanbari, A.: An adaptive network-based fuzzy inference system for short-term natural gas demand estimation: Uncertain and complex environments. Energy Policy 38(3), 1529–1536 (2010). https:// doi. org/ 10. 1016/j. enpol. 2009. 11. 036. (Security, prosperity and community—towards a common european energy policy? Special section with regular papers) 5. Baker, D.: Effect of observation time on mean temperature estimation. J. Appl. Meteorol. 14(4), 471–476 (1975). https:// doi. org/ 10. 1175/ 1520- 0450(1975) 014< 0471: EOOTO M>;2. 0. CO;2 6. Baldacci, L., Golfarelli, M., Lombardi, D., Sami, F.: Natural gas consumption forecasting for anomaly detection. Expert Syst. Appl. 62, 190–201 (2016). https:// doi. org/ 10. 1016/j. eswa. 2016. 06. 013 7. Besse, P., Cardot, H., Stephenson, D.: Autoregressive forecasting of some climatic variations. Scand. J. Stat. 27, 673–687 (2000) 8. Bosq, D.: Modelization, non-parametric estimation and prediction for continuous time processes. In: Roussas, G. (ed.) Nonparametric Functional Estimation and Related Topics, NATO Science Series C, pp. 509–529. Springer, Berlin (1991) 9. Bosq, D.: Linear Processes in Function Spaces: Theory and Applications. Springer, New York (2000) 10. Brabec, M., Konár, O., Pelikán, E., Malý, M.: A nonlinear mixed effects model for the prediction of natural gas consumption by individual customers. Int. J. Forecast. 24(4), 659–678 (2008). https:// doi. org/ 10. 1016/j. ijfor ecast. 2008. 08. 005. (Energy forecasting) 11. Chen, X.: Large sample sieve estimation of semi-nonparametric models. Handb. Econ. 6, 5549– 5632 (2007) 12. Chen, Y., Chua, W.S., Koch, T.: Forecasting day-ahead high-resolution natural-gas demand and supply in Germany. Appl. Energy 228, 1091–1110 (2018) 13. Chen, Y., Li, B.: An adaptive functional autoregressive forecast model to predict electricity price curves. J. Bus. Econ. Stat. 35, 371–388 (2017) 14. Chen, Y., Marron, J.S., Zhang, J.: Modeling seasonality and serial dependence of electricity price curves with warping functional autoregressive dynamics. Ann. Appl. Stat. 13, 1590–1616 (2019) 15. Costa, O.L.V., de Oliveira Ribeiro, C., Ho, L.L., Rego, E.E., Parente, V., Toro, J.: A robust least square approach for forecasting models: an application to Brazil’s natural gas demand. Energy Syst. 87, 1868–3975 (2019). https:// doi. org/ 10. 1007/ s12667- 019- 00351-1 16. Degiannakis, S.: Multiple days ahead realized volatility forecasting: single, combined and average forecasts. Glob. Financ. J. 36, 41–61 (2018). https:// doi. org/ 10. 1016/j. gfj. 2017. 12. 002 17. Dombaycı, O.: The prediction of heating energy consumption in a model house by using artificial neural networks in denizli-turkey. Adv. Eng. Softw. 41(2), 141–147 (2010). https:// doi. org/ 10. 1016/j. adven gsoft. 2009. 09. 012 18. Elman, J.L.: Finding structure in time. Cogn. Sci. 14, 20 (1990)