Forecasting and Advanced Smart Grid Modeling to Enhance Electricity Infrastructure Resilience in Iraq

Forecasting and Advanced Smart Grid Modeling to Enhance Electricity Infrastructure Resilience in Iraq DOCTORAL THESIS Morteza Rahmah Mkhailef Aldarraji Department of Computer Languages and Systems Higher Technical School of Computer Engineering University of Seville Month and Year of Submission Forecasting and Advanced Smart Grid Modeling to Enhance Electricity Infrastructure Resilience in Iraq Morteza Rahmah Mkhailef Aldarraji Directed by Doctors Dr. Beatriz Pontes Balanza Dr. Belén Vega Márquez Department of Computer Languages and Systems Higher Technical School of Computer Engineering University of Seville Month and Year of Submission Copyright ©Morteza Rahmah Mkhailef Aldarraji This work has been funding by the Spanish Ministry of Science and Innovation under project PID2020-117954RB-C22 financed by MCIN/ AEI /10.13039/501100011033. This work has also been supported by by the Ministry of Electricity/Operation and Control Office/Baghdad/Iraq [Contract 22156] and the Andalusian Regional Government [PYC20 RE 078 US]. v Acknowledgments First and foremost, I am grateful to God, the Lord of the Worlds, for bestowing upon me the patience and ability to complete this work successfully. This thesis represents the culmination of a long study project that began approximately four years ago. Its genesis was driven by a growing fascination with fields such as artificial intelligence and machine learning. I am grateful to my country, Iraq, for providing me with the opportunity to pursue my academic interest in smart grids through a scholarship. The journey was beset with numerous obstacles, yet these challenges catalyzed my intellectual and personal growth, culminating in the submission of this thesis today. However, this achievement cannot be attributed to me alone. I am extremely grateful to the many individuals who have provided me with invaluable assistance and guidance, which has greatly impacted my growth and progress in both academic and professional endeavors. I am profoundly grateful to my supervisors, Beatriz and Belén, for their remarkable tolerance and understanding, which have been a significant source of inspiration for me. Their guidance has been instrumental in my personal development, fostering authenticity and practicality in my approach to study. Pepe has been an invaluable source of inspiration and a mentor, encouraging my creative thinking since our first meeting. Their unwavering assistance and precise guidance have proven to be extremely helpful. My colleagues from the University of Seville and the University of Granada played a pivotal role in this endeavor. I am grateful for the intellectually stimulating discussions and their willingness to engage in ongoing discourse. I would like to express my deep gratitude to the faculty and staff of the Department of Science and Information Technology at the University of Granada for their warm hospitality and unwavering support throughout my time there. I would like to express my gratitude to the deceased Ali Fadel, who provided me with invaluable support and encouragement throughout my academic pursuits. Despite your departure from this world, your legacy lives on in the hearts of those who knew you. I write these words while experiencing a profound sense of loss and longing for your presence. vii viii Acknowledgments I would also like to express my gratitude to all my friends who tolerated me as I navigated the complexities of life, including its challenges and triumphs. I am grateful to Ehab and Ahmed, who played a pivotal role in my academic and personal growth. I would also like to express my gratitude to Dr. Basim from the University of Mosul in Iraq, who never tired of asking me questions while I worked and answered my questions even late at night. To my grandmother, I informed her that you left this world while I was engaged in academic pursuits. Nevertheless, you remain in our hearts, even though I miss you, your prayers, and your mellifluous voice. My family members have been of great assistance to me. My mother’s love is limitless. My father instilled in me a passion for knowledge and a desire to pursue academic excellence. My wife has been a constant companion, particularly during the challenges of welcoming our fourth child, Ayham. To my children, Jaafar, Ali, and Ibrahim, as well as to my brothers, especially Mohanad, I am grateful for their unwavering support and inspiration. Each of these individuals has made a significant contribution to my academic development, for which I am extremely grateful. This thesis not only demonstrates my own achievements but also provides evidence of the collective support and inspiration that have enabled me to reach this point. Abstract This dissertation addresses the pressing challenge of electricity supply and demand management in Iraq, a country whose energy sector is beset by infrastructural damage, insufficient investment, and increasing demand due to population and economic growth. This study is driven by the significant discrepancy between the rapidly growing energy demand, which has increased from 11,000 MW in 2007 to 16,000 MW in 2013. Concurrently, the demand increased to 24,500 MW in 2018 and has recently reached approximately 30,000 MW. Furthermore, the inadequate production of electricity has been exacerbated by successive wars and neglect. This study focuses on the Iraqi Electricity Power Grid (IEPG) and employs advanced modeling and artificial intelligence techniques to address the challenges arising from the recent surge in electricity demand. It examines forecasting energy demand and supply, assesses the network’s current performance, and proposes robust models for its enhancement. The study employs an unprecedented dataset from 2019 to 2021 provided by the Iraqi Ministry of Electricity to predict electricity demand and supply over varying horizons (24, 48, 72, and 168 hours ahead). This dataset was utilized in conjunction with a diverse array of advanced forecasting models, including linear regression, XGBoost, random forest, long short-term memory (LSTM), temporal convolutional networks (TCN), and multi-layer perceptron (MLP), in order to achieve the most accurate predictions possible. The results of numerous studies have consistently demonstrated that linear regression is more effective than other methods in demand forecasting. Conversely, XGBoost has been shown to be particularly adept at supply predictions. This comprehensive evaluation across different models and forecast horizons not only demonstrates the intricate capabilities of each method but also emphasizes the necessity of employing a multifaceted approach to enhance forecasting accuracy in a complex energy landscape like Iraq’s. This work represents a pioneering effort in applying complex network (CN) concepts to model the IEPG network. This work used two datasets: the first represents the power supply and demand, and the second represents the data about the power stations and the details about the IPG network. This offers a novel perspective on the intricate grid of generation power stations, ix xvi Índex of Figures 3.17. Consensus Clustering Process [66]. . . . . . . . . . . . . . . . 64 5.1. Computing the input gate, forget gate, and output gate in an LSTMmodel[197]......................... 77 5.2. Architectural elements in a TCN. (a) A dilated causal convolution with dilation factors d= 1,2,4and filter size k= 3. The receptive field is able to cover all values from the input sequence. (b) TCN residual block. An 1×1convolution is added when residual input and output have different dimensions. (c) An example of residual connection in a TCN. The blue lines are filters in the residual function, and the green lines are identity mappings [27]. . . . . . . . . . . . . . . . . . 79 5.3. Structure of MLP [201]. . . . . . . . . . . . . . . . . . . . . . 84 5.4. Linear regression with nonlinear variable transformations. (A) A nonlinear relationship, whose strength is underestimated by a linear least squares fit; (B) the same data recoded, which is much more suitable for linear regression [80]. . . . . . . . . . 85 5.5. Flow chart of the random forest algorithm [192]. . . . . . . . 86 5.6. The block diagram of the WR-XGBoost model [40]. . . . . . . 88 6.1. Directed Graph [52] . . . . . . . . . . . . . . . . . . . . . . . 93 6.2. Undirected Graph [52] . . . . . . . . . . . . . . . . . . . . . . 93 6.3. Weighted Graph [52] . . . . . . . . . . . . . . . . . . . . . . . 93 6.4. Simple Graph [159] . . . . . . . . . . . . . . . . . . . . . . . . 94 6.5. Multiple Graph [126] . . . . . . . . . . . . . . . . . . . . . . . 94 6.6. Complete Graph [92] . . . . . . . . . . . . . . . . . . . . . . . 94 6.7. Visualization of a segment of the IPG network showing power stations and their interconnections [10]. . . . . . . . . . . . . 96 7.1. SOM Clustering Algorithm [151]. . . . . . . . . . . . . . . . . 103 7.2. K-means Clustering Algorithm [103]. . . . . . . . . . . . . . . 104 7.3. Consensus Clustering Process [90]. . . . . . . . . . . . . . . . 104 8.1. Power demand and supply in Iraq for 2019, 2020, and 2021. . 112 8.2. Demand vs supply during the different seasons. . . . . . . . . 113 8.3. Demand vs supply on national public holidays. . . . . . . . . 114 8.4. Demand vs supply during Ramadan and Sha’ban periods. . . 114 8.5. Dickey-Fuller Test for Demand. . . . . . . . . . . . . . . . . . 115 8.6. Dickey-Fuller Test for Supply. . . . . . . . . . . . . . . . . . . 115 8.7. Hyperparameter optimization process. . . . . . . . . . . . . . 118 8.8. Demand trend and seasonality analysis. . . . . . . . . . . . . 125 8.9. Supply trend and seasonality analysis. . . . . . . . . . . . . . 125 Índex of Figures xvii 8.10. Distribution of results in terms of MAE grouped by forecast horizon. ..............................131 8.11. Comparison between actual and predicted values for demand in the last month of the data set. The observed results are those obtained with the best model. . . . . . . . . . . . . . . 131 8.12. Learning curves for Linear Regression (24h Forecast and 120h Pasthistory)............................132 8.13. Distribution of results in terms of MAE grouped by forecast horizon. ..............................137 8.14. Comparison between actual and predicted values for supply in the last month of the data set. The observed results are those obtained with the best model. . . . . . . . . . . . . . . . . . . 137 8.15. Learning curves for XGB (24h Forecast and 72h Past history). 138 8.16. Comparison of results in terms of MAE. The comparison was made between experiments, as shown in Tables 8.13 and 8.14, comparing the effect of including exogenous variables to perform the prediction. . . . . . . . . . . . . . . . . . . . . . . . 141 9.1. Power supply in Iraq for 2019, 2020, and 2021 in hourly basis. 146 9.2. Power demand in Iraq for 2019, 2020, and 2021 in hourly basis.146 9.3. Power supply in Iraq for 2019, 2020, and 2021 in weekly basis. 147 9.4. Power demand in Iraq for 2019, 2020, and 2021 in weekly basis.147 9.5. The gap between Power demand and supply in Iraq. . . . . . 147 9.6. Coordinates of the current IPG stations. . . . . . . . . . . . . 155 9.7. Visualization of the current IPG network model. . . . . . . . 155 9.8. Visualization of the IPG network after appending ten G.P.S. The original stations of IPG are represented in blue color, while the appended stations are shown in green color. Nodes size reflects the capacity of the stations (bigger nodes means higher capacity). . . . . . . . . . . . . . . . . . . . . . . . . . 157 9.9. Zoomed visualization of some G.P.S in Northern of Iraq. The original stations of IPG are represented in blue color, while the appended stations are shown in green color. Nodes size reflect the capacity of the stations (bigger nodes means higher capacity). .............................158 9.10. Zoomed visualization of some G.P.S in Middle of Iraq. The original stations of IPG are represented in blue color, while the appended stations are shown in green color. Nodes size reflect the capacity of the stations (bigger nodes means higher capacity). .............................158 9.11. Visualization of the IPG network after appending UHV-400 (red color nodes) and H.V.G. (black color nodes). The blue nodes represent the original stations of IPG network. . . . . . 159 xviii Índex of Figures 9.12. Zoomed visualization of the IPG network after appending UHV-400 (red color nodes) and H.V.G (black color nodes) in Northern of Iraq. The grey color nodes represent the original stationsinIPG...........................159 9.13. Visualization of the IPG network after appending UHV-400 (red color) and H.V.G (black color) in middle of Iraq. The grey color nodes represent the original stations in IPG. . . . . 160 9.14. Visualization of the IPG network after added HV-132. The red color nodes represent the original stations in IPG, while the black nodes represent the added stations. . . . . . . . . . 160 9.15. Zoomed visualization of the IPG network after added HV-132 (black nodes) in Northern of Iraq. The red nodes represent the original stations in IPG. . . . . . . . . . . . . . . . . . . . 161 9.16. Visualization of the IPG network after added HV-132 (black nodes) in middle of Iraq. The red nodes represent the original stationsinIPG...........................161 9.17. Visualization of the IPG network after added HV-132 (black nodes) in Southern of Iraq. The red nodes represent the original stations in IPG. . . . . . . . . . . . . . . . . . . . . . . . 162 10.1. Temperature Time Series . . . . . . . . . . . . . . . . . . . . 168 10.2. Power Demand series . . . . . . . . . . . . . . . . . . . . . . . 169 10.3. Power Supply series . . . . . . . . . . . . . . . . . . . . . . . . 170 10.4. temprature, demand, supply series . . . . . . . . . . . . . . . 171 10.5. DTW Averaged SOM Clustering of Temperature Data. . . . . 178 10.6. Temperature Data Clustering: SOM Histogram Distributions. 178 10.7. K-Means Clustering of Temperature Data. . . . . . . . . . . . 178 10.8. Temperature Data Clustering: k-mean Histogram Distributions.178 10.9. Consensus Hierarchical Clustering of Iraqi Cities Based on TemperatureData.........................179 10.10.DTW Averaged SOM Clustering of Demand Data. . . . . . . 180 10.11.Demand Data Clustering: SOM Histogram Distributions. . . . 180 10.12.K-Means Clustering of Demand Data. . . . . . . . . . . . . . 180 10.13.Demand Data Clustering: k-mean Histogram Distributions. . 180 10.14.Consensus Hierarchical Clustering of Iraqi Cities Based on DemandData. ..........................181 10.15.DTW Averaged SOM Clustering of Supply Data. . . . . . . . 182 10.16.Supply Data Clustering: SOM Histogram Distributions. . . . 182 10.17.K-Means Clustering of Supply Data. . . . . . . . . . . . . . . 182 10.18.Supply Data Clustering: k-mean Histogram Distributions. . . 182 10.19.Consensus Hierarchical Clustering of Iraqi Cities Based on SupplyData. ...........................183 Índex of Figures xix 10.20.DTW Averaged SOM Clustering of Temperature ,Supply and DemandData. ..........................184 10.21.Temperature ,Supply and Demand Data Clustering: SOM Histogram Distributions. . . . . . . . . . . . . . . . . . . . . . . . 184 10.22.K-Means Clustering of Temperature ,Supply and Demand Data.184 10.23.Temperature ,Supply and Demand Data Clustering: k-mean Histogram Distributions. . . . . . . . . . . . . . . . . . . . . . 184 10.24.Consensus Hierarchical Clustering of Iraqi Cities Based on Temperature ,Supply and Demand Data. . . . . . . . . . . . . 185 10.25.Comparative Histograms of City Based on Temperature, Power supply and Demand profiles Across Clustering Segments . 186 10.26.Clusters Means in Temperature Data. . . . . . . . . . . . . . 186 10.27.Clusters Means in Demand Data. . . . . . . . . . . . . . . . . 187 10.28.Clusters Means in Supply Data. . . . . . . . . . . . . . . . . . 187 List of Tables 1.1. Addressing Energy Challenges in Iraq: Forecasting Power Supply and Demand Using Artificial Intelligence Models . . . . . . . 14 1.2. Clustering Cities Based on Temperature, Power Supply, and Power Demand: Implications for Energy Sector Planning in Iraq 16 1.3. Certificate of Attendance for Morteza Rahmah Mkhailef Aldarraji at the COMPASS Conference . . . . . . . . . . . . . . 17 1.4. Time Series Analysis of Iraq’s Energy Sector: Production, Consumption, and Trends . . . . . . . . . . . . . . . . . . . . 18 2.1. Comparison between conventional and smart grids. . . . . . . 32 3.1. Current types of power stations (GS) used in the Iraqi power gridnetwork. ........................... 51 3.2. Distributions of stations in Iraq and their details. . . . . . . . 51 3.3. Overview and Comparative Study of SOM, K-means, and Consensus Clustering Algorithms . . . . . . . . . . . . . . . . 65 4.1. Summary of the threat to validate of this work. . . . . . . . . 74 5.1. A summary of the literature. . . . . . . . . . . . . . . . . . . 89 6.1. A summary of the literature. . . . . . . . . . . . . . . . . . . 100 7.1. A summary of the literature. . . . . . . . . . . . . . . . . . . 108 8.1. Training parameters used for deep learning models. . . . . . . 119 8.2. The parameters used for LSTM, TCN, and MLP deep learning models. ..............................120 8.3. The parameters used for Random Forest, Linear Regression, and XGBoost machine learning models. . . . . . . . . . . . . 121 8.4. The best parametrization for each model and dataset. . . . . 122 8.5. Forecasting demand with a forecast horizon of 24h and different past history scenarios. The optimal outcomes of each model have been emphasized in bold. . . . . . . . . . . . . . . 127 xxi xxii Índex of Tables 8.6. Forecasting demand with a forecast horizon of 48h and different past history scenarios. The optimal outcomes of each model have been emphasized in bold. . . . . . . . . . . . . . . 128 8.7. Forecasting demand with a forecast horizon of 72h and different past history scenarios. The optimal outcomes of each model have been emphasized in bold. . . . . . . . . . . . . . . 129 8.8. Forecasting demand with a forecast horizon of 168h and different past history scenarios. The optimal outcomes of each model have been emphasized in bold. . . . . . . . . . . . . . . 130 8.9. Forecasting supply with a forecast horizon of 24h and different past history scenarios. The optimal outcomes of each model have been emphasized in bold. . . . . . . . . . . . . . . . . . . 133 8.10. Forecasting supply with a forecast horizon of 48h and different past history scenarios. The optimal outcomes of each model have been emphasized in bold. . . . . . . . . . . . . . . . . . . 134 8.11. Forecasting supply with a forecast horizon of 72h and different past history scenarios. The optimal outcomes of each model have been emphasized in bold. . . . . . . . . . . . . . . . . . . 135 8.12. Forecasting supply with a forecast horizon of 168h and different past history scenarios. The optimal outcomes of each model have been emphasized in bold. . . . . . . . . . . . . . . 136 8.13. Results in terms of MAE and MSE for the data sets using demand only. For each dataset the best result is underlined in bold.................................140 8.14. Results in terms of MAE and MSE for the data sets using demand and exogenous features. For each dataset the best result is underlined in bold. . . . . . . . . . . . . . . . . . . . 140 9.1. IPG Dataset attributes and description. . . . . . . . . . . . . 148 9.2. Statistical characteristics of Power Grids. . . . . . . . . . . . 151 9.3. The Adopted proposed Scenarios. . . . . . . . . . . . . . . . . 154 9.4. Comparison between all proposed scenarios and the current IPGnetwork. ...........................162 10.1. Overview and Comparative Study of SOM, K-means, and Consensus Clustering Algorithms. . . . . . . . . . . . . . . . . 173 Part I Introduction Chapter 1 Background In the dynamic realm of technology and society, the pursuit of efficiency and optimization has consistently fueled innovation. As a result of this endeavor, several tools and approaches have been created to enhance our capacity to carry out tasks, automate processes, and improve decision-making procedures. Computing has become a fundamental aspect of technological progress, revolutionizing the way we acquire, manipulate, and use data. The history of computing demonstrates humanity’s unwavering pursuit to surpass constraints, beginning with the ancient practice of using clay tokens to record transactions. These tokens were used primarily by ancient civilizations that flourished in the Mesopotamian region (currently Iraq), including the Sumerians, Babylonians, and Assyrians. This practice evolved into the development of advanced computer systems that characterize the information age. [164]. The rapid advancement of automation and data storage optimization has completely transformed how we interact with information. These innovations enable us to instantly access vast data warehouses and automate complex analytical operations. These technological developments have laid the foundation for the rise of artificial intelligence (AI), a discipline focused on replicating human intellect to address challenges in many fields. AI encompasses several disciplines that aim to imitate cognitive abilities, with machine learning being a particularly important area of emphasis [162]. Machine learning leverages the capacity to extract valuable information from data, automating decision-making procedures and significantly improving the ability to tackle complex problems without requiring extensive domain knowledge. This dissertation delves into an advanced segment of machine learning known as deep learning [187], inspired by the neural networks of the human brain. Deep learning algorithms have demonstrated impressive adaptability and effectiveness in a wide range of applications, including image identification and natural language processing, by imitating brain connections and information processing. This study focuses on the use of deep learning in 3 10 Chapter 1. Background its empirical analysis of various machine learning and deep learning models to predict electricity supply and demand. It also distinguishes itself from the existing literature by addressing the limitations of previous works and offering insights into the unique challenges faced in Iraq’s energy sector. The intricacies within the IPG network pose challenges, particularly in densely populated regions. While various studies have introduced approaches, they often lack a comprehensive, in-depth investigation. Additionally, the expansion of the IPG based on complex networks has not been explored in the literature. More importantly, modeling the IPG has not been attempted in previous works. Here, the modeling approach is a crucial procedure that can be used to address the issues facing Iraq’s power grid network. By developing a comprehensive model of the network, it is possible to identify areas that require upgrades or repairs, optimize the distribution of power, and predict future demand. This can help to improve the reliability and efficiency of the network, reduce the frequency of blackouts and power cuts, and ensure that Iraq’s growing energy needs are met. Hence, the contributions of this work are as follows: Collecting a time-series-based dataset for the years 2019 to 2021, encompassing a range of electricity demand and supply values. The dataset is novel and was officially collected with the support of the Operation and Control Office, Ministry of Electricity, Baghdad, Iraq. The minimum electricity demand value is 6,336 MW/day, and the maximum value is 29,059 MW/day. The minimum supply value is 5,399 MW/day, and the maximum value is 18,233 MW/day. Using the collected dataset, the supply and demand of electricity in 15 provinces in Iraq were predicted. The structural time-series modeling approach was applied to annual data for the period between 2019 and 2021, using estimated equations and value assumptions. To make the predictions, the study used a range of machine learning and deep learning models, including Linear Regression (LR), XGBoost (XGB), Random Forest (RF), Long Short-Term Memory (LSTM), Temporal Convolutional Networks (TCN), and Multilayer Perceptron (MLP) models. The study also used various metrics for benchmarking and statistical analysis to verify the differences between the involved models. Generating a comprehensive network model for the Iraqi Power Grid (IPG) to facilitate the simulation of various real-life scenarios. These scenarios, designed to address core issues within the IPG, will leverage detailed network performance analysis based on the principles of complex networks (CN). Each scenario aims to mitigate challenges identified by experts from the Iraqi Ministry of Electricity, thereby reflecting authentic situations that the IPG might encounter. This in- 1.3. Objectives 11 tegrated approach ensures that the model serves as a critical tool in strategic planning and operational adjustments. Application of Advanced Clustering Techniques: Employing the newly acquired dataset, we apply advanced clustering techniques to scrutinize the temperature, supply, and demand dynamics of electricity across 15 provinces in Iraq. To achieve this, we leverage the Self-Organized Map (SOM) and K-means algorithms for benchmarking purposes. 1.3. Objectives The main objectives of the dissertation are summarized as follows: 1. Address Global and Local Energy Demand Issues: The study aims to present a comprehensive understanding of the global issue of energy demand, including its impact on the country of the case study. It seeks to delve into energy forecasting, clustering, and developing strategies (network modeling) to improve domestic power supply and reduce reliance on external sources. 2. Explore Forecasting Models, Clustering, and Network Modeling: Before diving into specific solutions, the work intends to explain common forecasting models, clustering techniques, and strategic approaches to tackle energy demand and supply issues effectively. 3. Identification and Utilization of Advanced Clustering Techniques: Through the application of methods like the Self-Organized Map (SOM) and K-means algorithms, the study plans to uncover insights by clustering cities into distinct groups based on their energy dynamics and needs. This facilitates targeted infrastructure planning and policy formulation that are specific to the energy requirements of different regions. 4. Contribute to a Sustainable Energy Landscape: The study seeks to contribute to the development of a sustainable and resilient energy framework by leveraging advanced clustering techniques and forecasting models. This includes addressing immediate energy needs and laying the foundation for a more adaptable energy system in the future. 5. Focus on Iraq’s Specific Energy Challenges: The study specifically targets the energy demand issue in Iraq, considering its unique challenges such as security instability, infrastructure mismanagement, and the need for technological upgrades. It aims to provide a detailed analysis of Iraq’s power grid and propose strategies to improve its efficiency and reliability. 12 Chapter 1. Background 6. Predict Electricity Supply and Demand with Machine Learning and Deep Learning Models: Employing a novel time-series-based dataset, the study aims to predict the electricity supply and demand across 15 provinces in Iraq using various machine learning and deep learning models. This objective is central to designing more efficient energy distribution strategies and addressing gaps in current forecasting practices. 7. Generate and Simulate Network Models for the Iraqi Power Grid (IPG): The work plans to create a comprehensive network model for the IPG and use this model to simulate real-life scenarios. This simulation aims to improve understanding and address the core issues within the IPG, thereby enhancing its performance and reliability. 8. Include Sociodemographic Factors in Prediction Models: The study highlights the importance of incorporating sociodemographic factors such as seasons, national holidays, and cultural events (e.g., Ramadan and Sha’ban) to enhance prediction accuracy. This approach recognizes the significant impact these factors have on energy consumption patterns. 9. Comprehensive Review and Innovation in Energy Forecasting: By conducting a thorough literature review and employing innovative forecasting methods, the study aims to bridge the gap in current research on energy forecasting and clustering techniques, specifically in the context of Iraq’s energy sector. 1.4. Thesis Organization The thesis is organised into the following eleven chapters: Chapter 1: General introduction to deep learning and machine learning techniques for time series forecasting to enhance the Iraqi Power Grid’s reliability, efficiency, and resilience, addressing challenges in energy supply and demand management within a smart grid framework. Chapter 2: Presents essential concepts and theories, focusing on machine learning, smart grid technologies, and network analysis to improve Iraq’s power infrastructure. It covers the evolution and significance of smart grids, the role of communication technologies like ZigBee and WLAN, and the integration of AI. The chapter also addresses energy demand and supply dynamics, forecasting methods, and complex network analysis to enhance energy system performance and robustness. 1.4. Thesis Organization 13 Chapter 3: Energy Systems and Forecasting, covering topics such as global and Iraqi energy demand, energy forecasting, power manufacturing, various energy resources, power supply and demand challenges, the Iraqi power grid, time series analysis, and complex networks. Chapter 4: Addresses the state of the art in power system management and forecasting, and presents an analysis of recent studies on predictive models for power supply and demand. These include Long Short-Term Memory Networks, Temporal Convolutional Networks, Multi-layer Perceptrons, Linear Regression, Random Forest, and XGBoost. Each model is supported by a comprehensive review of the relevant literature. Chapter 5: Presents a comprehensive review of network modeling techniques, with a particular focus on the challenges posed by the intricate structures of power grids. Chapter 6: Examines clustering methods such as Self-Organized Maps and K-Means, presenting discussions on methodology and practical implications, with a focus on related research. Chapter 7: Models of predicting power supply and demand, detailing the methodology including data exploration, time series and deep learning models, experimental setup, and evaluation metrics, followed by results on energy demand and supply forecasting, demand prediction, discussions, and conclusions. Chapter 8: Network Modelling, which includes topics such as datasets for power supply and demand, IPG network datasets, cumulative datasets for IPG modeling, network modeling, adopted scenarios, inspections of IPG development and expansion, implementing and evaluating scenarios, results visualization, scenario performance evaluation, feasibility, and conclusions. Chapter 9: Clustering Techniques, covering topics such as the Self- Organized Map (SOM) algorithm, K-means algorithm, consensus clustering, experimental setup, and various results of clustering based on temperature, power demand, power supply, and their combinations. It also includes sections on cluster distributions, cluster means and characteristics, discussion, and conclusion. Chapter 10: Conclusions Chapter 11: The final chapter of the thesis is titled "Future Works" 14 Chapter 1. Background 1.5. Publications The main contributions of this thesis, both already published and in the process of being published and under modification, are listed below: 1. Addressing Energy Challenges in Iraq: Forecasting Power Supply and Demand Using Artificial Intelligence Models Journal Heliyon Year 2024 Pages 930-935 Volume 10 DOI 10.1016/j.heliyon.2024.12345 IF 4.567 (JCR 2024) Ranking Energy (24/114 - Q1) Citations -(Scopus); -(Google Scholar); -(Web of Science) Table 1.1: Addressing Energy Challenges in Iraq: Forecasting Power Supply and Demand Using Artificial Intelligence Models Abstract The global surge in energy demand, driven by technological advances and population growth, underscores the critical need for effective management of electricity supply and demand. In certain developing nations, a significant challenge arises because the energy demand of their population exceeds their capacity to generate, as is the case in Iraq. This study focuses on energy forecasting in Iraq, using a previously unstudied dataset from 2019 to 2021, sourced from the Iraqi Ministry of Electricity. The study employs a diverse set of advanced forecasting models, including Linear Regression, XGBoost, Random Forest, Long Short-Term Memory, Temporal Convolutional Networks, and Multi-Layer Perceptron, evaluating their performance across four distinct forecast horizons (24, 48, 72, and 168 hours ahead). Key findings reveal that Linear Regression is a consistent top performer in demand forecasting, while XGBoost excels in supply forecasting. Statistical analysis detects differences in models performances for both datasets, although no significant differences are found in pairwise comparisons for the supply dataset. This study emphasizes the importance of accurate energy forecasting for energy security, resource allocation, and policy-making in Iraq. It provides tools for decision-makers to address energy challenges, mitigate power shortages, and stimulate economic growth. It also encourages innovative forecasting methods, the use of external variables like weather and economic data, and region-specific models tailored to Iraq’s energy landscape. The research contributes 1.5. Publications 15 valuable insights into the dynamics of electricity supply and demand in Iraq and offers performance evaluations for better energy planning and management, ultimately promoting sustainable development and improving the quality of life for the Iraqi population. 16 Chapter 1. Background 2. Iraqi Power Grid Developing Strategies: A Network Model Abstract The recent extensive damage to the Iraqi Power Grid (IPG) infrastructure has brought significant challenges to power supply and demand. It has resulted in a substantial decrease in power supply, reaching up to 17,000 MVA, while the actual demand stands at 27,000 MVA. The gap between power production and power supply is considered one of the challenges of the Iraqi government. One of the solutions to fill this gap is expanding the power grid. However, the expansion of the power grid is subject to many factors that should be taken into considerations. One of these factors is the structure of the grid after performing the expansion (e.g., adding power stations), which should maximize the performance of the network. In this work, various scenarios have been put forth to improve the performance of IPG. Each proposed scenario aims to address a specific challenge that might be encountered in the network. The scenarios are visualized using concepts inspired by Complex Networks and then analyzed and benchmarked. Several network metrics have been involved in each scenario for performance evaluation. The results showed that the proposed scenarios are feasible to be adopted and can improve the whole performance of the IPG network model. 3. Clustering Cities Based on Temperature, Power Supply, and Power Demand: Implications for Energy Sector Planning in Iraq Journal In the process of being submitted Year - Pages - Volume - DOI - IF - Ranking - Citations - Table 1.2: Clustering Cities Based on Temperature, Power Supply, and Power Demand: Implications for Energy Sector Planning in Iraq Abstract This study presents a novel approach to understanding the complex interplay between climatic factors and energy requirements by clustering 15 Iraqi cities based on temperature, power supply, and power demand. Utilizing a rich dataset spanning 2019 to 2020, the research aimed to unveil patterns and correlations that could inform 1.5. Publications 17 efficient energy sector planning and policy-making in Iraq. The methodology encompassed rigorous data preprocessing, including temporal data integration, removal of non numeric symbols, forward-filling missing temperature data, and standardization. The study employed three distinct clustering techniques: Self-Organizing Maps (SOM), K-means algorithm, and Consensus Clustering. SOM was chosen for its proficiency in identifying non-linear relationships, while K-means offered a straightforward approach to partitioning data into clusters. Consensus Clustering combined these methods to enhance the robustness of the findings. Our results revealed significant regional variations in temperature, power supply, and demand. Clusters formed highlighted provinces with similar energy profiles, suggesting the potential for tailored energy policies. The study also identified provinces with erratic energy patterns, underscoring areas needing focused intervention. These insights are pivotal for developing strategic energy plans that consider both climatic and energy demand fluctuations. This research contributes to the growing field of geographical energy studies, offering a comprehensive framework that integrates climatic and energy data for strategic planning. The findings hold particular relevance for countries like Iraq, where climate and energy are crucial to economic and social stability. 4. COMPASS Conference Certificate Type Certificate of Attendance Participant Morteza Rahmah Mkhailef Aldarraji Event COMPASS Conference Location Helsinki Date 4-5 October 2023 Organizer Ulysseus European University Issued on 5 October 2023 Signature Dr. Kitte Marttinen, Haaga-Helia Ulysseus Project Director Table 1.3: Certificate of Attendance for Morteza Rahmah Mkhailef Aldarraji at the COMPASS Conference Abstract Urban energy management is increasingly complex due to fluctuating demand and diverse power supply sources. This study presents an innovative approach to urban planning and energy resource allocation through the clustering of cities based on three key variables: temperature, power supply, and power demand. Employing Self-Organized Map (SOM) and K-means clustering algorithms, we analyze data from 15 Iraqi provinces, collected between 2019 and 2020. Our methodology involves detailed preprocessing of time-series data, including the 18 Chapter 1. Background integration of disparate data forms into a consistent format and the application of dynamic time warping to handle temporal variations. The research reveals distinct clusters of cities with similar energy characteristics, enabling targeted policy-making and infrastructure development. By identifying unique patterns of energy usage and demand, our study provides crucial insights into efficient resource allocation and strategic energy planning, enhancing the resilience of urban energy systems against climatic and supply variability. This clustering approach not only informs local energy policies but also contributes to broader discussions on sustainable urban planning. 5. Time Series Analysis of Iraq’s Energy Sector: Production, Consumption, and Trends Event Congress Location Slamanca Date October 2024 Pages 10 Organizer Salamanca University Table 1.4: Time Series Analysis of Iraq’s Energy Sector: Production, Consumption, and Trends Abstract This study is motivated by Iraq’s dependence on external energy sources despite having diverse energy options. The main aim is to predict energy demand based on past data, also including unique events like local festivals or critical periods of time. LSTM networks are chosen for their memory feature, enabling predictions based on past time periods. Data from 2019 to 2022 on energy production and demand in each province of Iraq is utilized, where the dataset has been studied by hours, days, and months in search of better predictions. Different ways of training the model have been explored, including different parametrization, groupings of the demand data and additional characteristics, culminating in the interpretation of the best prediction. The incorporation of additional sociodemographical information to the model has been proved to enhance its precision, and the obtained results align with Iraq’s macroeconomic landscape, confirming the appropriateness of incorporating various additional timestamp features, such as seasons, critical periods and holidays to the prediction model. Chapter 2 Smart Grids and AI/ML Technologies This thesis explores the advanced utilization of artificial intelligence (AI) in energy management systems, with a focus on the strategic implementation of machine learning (ML) and deep learning (DL) methods for advanced time series forecasting. The study begins with a thorough examination of the current integration of AI into smart grid technologies. This integration has a substantial impact on improving the operational efficiency, flexibility, and resilience of electric power distribution systems. This connection enables the timely analysis of data and the ability to make informed decisions in realtime. It also supports the implementation of proactive maintenance strategies, which are crucial for maintaining a strong and reliable energy infrastructure. The opening sections provide a comprehensive analysis of the literature that extensively discusses the transformative impact of machine learning and deep learning on energy distribution. These AI approaches improve algorithms that accurately forecast variations in power supply and demand, maximize the incorporation of renewable energy, and effectively control distributed energy resources with exceptional precision. The thesis also examines emerging AI trends in the energy industry, including sophisticated neural network structures for predicting energy demand and detecting abnormalities in grid operations, improving energy storage technologies. These advancements are crucial for achieving sustainability and environmental goals. The transition to AI-powered smart grids is presented as more than just a technological advancement; it is an essential shift in thinking to tackle the complexities of contemporary energy needs and the urgent issue of climate change. This transformation is supported by a robust framework of Internet of Things (IoT) devices, edge computing, and big data analytics. This framework creates a synergistic ecosystem that converts old grid systems into 19 26 Chapter 2. Smart Grids and AI/ML Technologies renewable energy sources by incorporating intelligence into the grid. This integration is essential as it transitions the conventional centralized power generating paradigm to a decentralized and flexible approach. SG technology is positioned at the convergence of energy sustainability and advanced grid management, with the potential to fundamentally transform the operations and interactions of power networks with renewable energy sources. Moreover, the process of digitizing the electric network that is involved in the implementation of Smart Grids highlights a more extensive technical transition towards the Internet of Energy (IoE). This idea represents a future in which energy production, distribution, and consumption are closely connected to digital communication technology. This connection will lead to highly efficient operations, flexible functioning, and increased user involvement in the energy sector. Nevertheless, the journey towards fully harnessing the capabilities of smart grids (SGs) and integrating renewable energy sources is hindered by several technological, legislative, and operational obstacles. The transition towards a smart, renewable-powered grid requires a collaborative effort from engineers, politicians, and the global community to ensure grid stability in the face of changing renewable outputs and address cybersecurity issues in an increasingly digital energy economy. The merger of SGs and RESs signifies more than simply a technological improvement in physical infrastructure. It entails a complete reinvention of the future of the energy industry. The vision entails a global scenario where energy is not only cleaner and more sustainable, but also intelligently controlled and distributed. This represents a substantial advancement in our joint pursuit of environmental responsibility and energy adaptability as Figure 2.3. Figure 2.3: The transition of power grid towards smart grid with diversification and distributed generation [94]. 2.4. The smart grid concept and communication technologies 27 The smart grid’s communication infrastructure can rely on three network types: Home Area Network (HAN), Neighborhood Area Network (NAN), and Wide Area Network (WAN) [28]. Figure 2.4 displays the schematic architecture of the smart grid communication infrastructure that relies on these networks. HAN is implemented and managed within a limited range (often tens of meters), typically confined to a residential dwelling or a small workplace. The HAN has a comparatively lower transmission data rate in comparison to the other two networks, amounting to hundreds of bits per second (bps). A common implementation of a Home Area Network (HAN) is a broadband Internet connection that is shared among numerous users using either a wired or wireless modem. It facilitates the exchange of information and the use of resources among computers, mobile devices, and other devices through a network connection. Smart grid installation involves connecting all energy-consuming smart home gadgets and smart meters to the Home Area Network (HAN). The data from the devices is collected and transferred via HAN to the smart meters. HAN enables more effective management of energy use in residential settings. Home Area Network (HAN) can be deployed using either ZigBee or Ethernet technology.NAN is implemented and managed within a limited region of a few metropolitan structures, spanning hundreds of meters. Multiple Home Area Networks (HANs) can be linked to a single Neighborhood Area Network (NAN), facilitating the transmission of energy consumption data from each individual residence to the NAN network. The NAN network transmits this data to Local Data Centers for the purpose of storage. Data storage is crucial for billing customers and analyzing data to identify patterns in energy generation and consumption.The NAN has a maximum transmission data rate of 2 Kbps. The NAN may be deployed via Programmable Logic Controllers (PLC), Wi-Fi, and cellular technologies. The Wide region Network (WAN) is implemented and managed throughout a large geographical region spanning tens of kilometers. It is composed of many Neighborhood Area Networks (NANs) and Local Data Centers (LDCs). Furthermore, the interconnection of all components of the smart grid, such as the operator control center, main and renewable energy generation, transmission, and distribution, relies on a Wide Area Network (WAN). The Wide Area Network (WAN) has a significantly high transmission data rate, reaching up to several Gigabits per second (Gbps). The Wide Area Network (WAN) can be established using many technologies, such as Ethernet networks, WiMAX, 3G/LTE, and microwave transmission as 2.4. 28 Chapter 2. Smart Grids and AI/ML Technologies Figure 2.4: Smart grid communication infrastructure [28]. 2.5. Communication technologies for smart grids Various technologies can be utilized in the implementation of the smart grid [149] as can be seen in Figure 2.5: Figure 2.5: Communication Technologies of Smart Grid. 2.5.1. Power Line Communication (PLC) Power line communication enables the transmission of data between devices using electrical power lines. PLC is achieved by superimposing a modulated carrier signal onto the power wires. The data rate of OFDM-based communication can reach several megabits per second. Power-line communication protocols exhibit significant variations in data rates and distance limitations. Carriers with a frequency range of around 100 - 200 kHz that are applied to high-voltage transmission lines can transmit one or two analog speech circuits, or telemetry and control circuits with a data rate corresponding to a few hundred bits per second. It is important to note that these circuits can 2.5. Communication technologies for smart grids 29 span over great distances, potentially covering several miles. Increased data speeds often correspond to reduced transmission distances.Various forms of powerline communications utilize distinct frequency bands. Due to the initial design of the power distribution system for transmitting AC electricity at standard frequencies of 50 or 60 Hz, power wire circuits have restricted capacity to handle higher frequencies. The establishment of high frequency communication over power lines is challenging owing to the attenuation of high frequency transmissions. The propagation issue poses a constraint on power line communications.In addition, communication signals may experience interference from high-order harmonics generated by non-linear loads found in the grid, such as rectifiers and inverters. To address this issue, it is necessary to employ sophisticated active filters. PLC, or Power Line Communication, is utilized in smart grid applications to establish communication between smart meters and the Local Data Concentrator (LDC) in the Neighborhood Area Network. Nevertheless, the connection between the LDC and other smart grid components, including the operator control center, generation, transmission, and distribution, is carried out using cellular networks.The PLC offers the benefit of a previously developed and widely implemented infrastructure, which effectively lowers the costs associated with installation. Drawbacks include the existence of elevated harmonics in the power lines that disrupt communication signals and the restricted frequency range for communication [158] [134]. Figure 2.6: Typical smart metering system infrastructure includes power line communication (PLC)[56]. 2.5.2. ZigBee ZigBee operates according to the guidelines set by the IEEE 802.15 standard. ZigBee is used in applications that require low transmission speed, 30 Chapter 2. Smart Grids and AI/ML Technologies extended battery longevity, economical implementation, and secure networking. Applications encompass wireless light switches, electrical meters equipped with in-home displays, traffic control systems, and various consumer and industrial equipment that necessitates the transmission of data over short distances at relatively modest speeds. The ZigBee network can accommodate a maximum of 60,000 devices for connectivity. ZigBee operates at a specified speed ranging from 20 to 250 kbs, making it ideal for transmitting periodic or intermittent data, or a single signal from a sensor or input device. ZigBee technology, as specified, aims to be more cost-effective and less complex compared to existing wireless personal area networks (WPANs) like Bluetooth or Wi-Fi. ZigBee networks are protected by 128-bit symmetric encryption keys.The "ZigBee Smart Energy"program enables the incorporation of smart meters into the ZigBee network alongside other devices [116]. This program enables smart meters to get data from interconnected devices and manage them. In addition, users have the ability to monitor their energy usage in real-time. Furthermore, it enables improved energy efficiency and the implementation of real-time dynamic pricing. ZigBee utilization in smart grid offers cost-effectiveness, compactness, and utilizes a relatively narrow bandwidth.The drawbacks of ZigBee include a limited battery life due to its small battery size, a small memory capacity, a restricted data transfer rate, and a poor computing capabilities. Additionally, the device’s use of unlicensed frequencies at 868 MHz and 2.4 GHz might potentially cause interference with other Wi-Fi, Bluetooth, and Microwave communications [25]. 2.5.3. Wireless Local Area Network(WLAN) A wireless local area network (WLAN) connects several devices using spread spectrum or orthogonal frequency division multiplexing (OFDM) [28]. Typically, it provides internet connectivity through an access point. This feature allows users to navigate within a limited geographical area while maintaining a continuous connection to the network. Contemporary WLANs mostly utilize the IEEE 802.11 specifications, which are commercially promoted as Wi-Fi. Wireless Local Area Networks (WLANs) have gained popularity in residential settings because of their simple installation process. Additionally, they are commonly seen in business buildings where they provide wireless connectivity to consumers. Due to its extensive global deployment, WLAN can be seamlessly incorporated into the smart grid. Wireless Local Area Network operates within the frequency range of 2.4 GHz to 3.5 GHz. WLAN has several benefits, including cost-effectiveness, widespread global availability, and the convenience of plug and play devices. One significant drawback of WLAN is its susceptibility to interference from other devices that operate on the same frequency as figure 2.4. 2.6. From traditional power system to smart grid 31 2.5.4. Cellular Networks Cellular networks are extensively implemented in the majority of nations and possess a well established infrastructure. In addition, they provide highspeed data transmission at rates of up to 100 Mbps. Hence, cellular networks can facilitate communication among various components and devices inside a smart grid. Various cellular communication technologies, including GSM, GPRS, 2G, 3G, 4G, and WiMAX, have been developed [150] [140]. WiMAX technology is highly compelling for the development of smart grids. The device operates on frequencies of 2.5 and 3.5 GHz, with a data exchange rate of 70 megabits per second and a coverage range of up to 50 kilometers. The WiMAX chips are embedded in smart meters distributed throughout the smart grid. The benefits of cellular networks include an already established infrastructure that covers a large region, fast data transmission rates, and the presence of security methods that are already integrated into cellular communication. An inherent drawback of cellular networks is their shared nature, as they are not exclusively dedicated to smart grid communications. This can pose a significant issue during an emergency situation involving the power system. 2.6. From traditional power system to smart grid In the power grid setup, the operator manages the distribution of power from generators to fulfill customer demand. Customers are referred to as consumers in this setup. Furthermore, communication flows in one direction from the operator to customers. This one way communication poses challenges for the operator in maintaining grid stability as precise consumption data is not readily available. To address this problem, efforts are underway to evolve the power grid into a grid by incorporating Information and Communication Technologies (ICT) that facilitate directional information exchange between operators and consumers. The distinction between a grid and a traditional power grid is depicted in figure below. In a grid configuration households are equipped with meters that allow for bidirectional data transmission between customers and operators. These meters not facilitate monitoring of energy consumption. Also relay this information back to the operator for strategic decision making aimed at sustaining grid stability leveraging insights from smart meters. Additionally there is a growing trend towards integrating energy sources into the power grid providing operators with diversified options, for electricity generation. In the grid we are seeing the integration of technologies such, as energy storage devices and electric vehicles[70]. 32 Chapter 2. Smart Grids and AI/ML Technologies Figure 2.7: Comparison of a traditional power grid and a smart grid [70]. The table below differentiates between conventional and SGs [21]. Table 2.1: Comparison between conventional and smart grids. Existing Grid Smart Grid Electromechanical Digital Unidirectional communication Bidirectional communication Central generation Distributed/scattered generation Small No. of sensors Large No. of sensors Manual monitoring Self/automated monitoring Manual restoration Self-healing Failures/faults and blackouts Highly Adaptive and intelligent islanding Limited/restricted control Pervasive control Small No. of customer choices Large No. of customer choices The below sections present the theoretical foundation essential for understanding the subjects addressed in this dissertation. It begins by examining the dynamics of power supply and demand, segueing into an analysis of the Iraqi Power Grid’s structure and function. The exploration then advances into the field of Artificial Intelligence, with a detailed review of various pivotal algorithms. Subsequently, the discussion transitions to Complex Networks, where it elaborates on fundamental concepts, methodologies for network generation and visualization, and techniques for evaluating network metrics. This chapter establishes a comprehensive theoretical base, crucial for navigating the complex themes explored throughout the dissertation. The comprehensive dissertation delves into the intricacies of Iraq’s energy sector, blending a detailed analysis of its current state with a robust theoretical background on global energy demand issues, with a keen focus on Iraq’s unique position. It outlines the major challenges facing Iraq, including the impact of geopolitical instability, an outdated infrastructure, and the growing gap between energy production and demand. The study emphasizes the pivotal role of energy for national success, highlighting the risks posed by 2.6. From traditional power system to smart grid 33 non-renewable sources to the climate and the imperative need for sustainable solutions such as hydroelectric, geothermal, wind, and solar power. Central to this examination is the exploration of smart grid systems as a means to address the inefficiencies of traditional grids, employing complex networks theory for a sophisticated analysis of power grid networks. This method emphasizes how important it is to accurately predict energy demand through strategic planning and allocating resources. It also suggests using advanced machine learning and deep learning models, especially Long-Short- Term Memory (LSTM) models, which are better at handling time series data. Addressing the specific challenges in Iraq, the dissertation reviews the country’s efforts to modernize its electricity sector amidst an unstable security landscape and the critical need to boost domestic energy production to lessen external dependencies. It critically analyzes the gaps in time series analysis of temperature, supply, and demand within Iraq’s power grid, advocating for innovative energy planning strategies. The work proposes a novel methodology for energy forecasting in Iraq, involving the collection of time-series-based data and the application of various machine learning and deep learning models. It highlights the utilization of advanced clustering techniques for analyzing electricity dynamics across different provinces, aiming at laying the groundwork for a more sustainable and adaptable energy system. The dissertation offers a comprehensive package of strategies for enhancing energy efficiency, investing in renewable resources, and policy reforms. It stresses the importance of international collaboration and technological innovation in bridging the current gaps and achieving a reliable, affordable, and environmentally friendly energy supply for Iraq, setting a path towards sustainable development in the face of global and local challenges. Chapter 3 Energy Systems and Forecasting 3.1. Energy demand Introducing the context of the dissertation it is necessary to present the global issue of energy demand and the issue in the country of the case study. Before talking about energy forecasting, clustering, and proposed scenarios adopted to improve domestic power supply and reduce dependence on outside sources, it is also important to explain the most common forecasting models, clustering, and strategies. This is necessary to accurately define the study problem and the methods used to tackle it. The matters at hand are thoroughly examined and resolved in this section. 3.1.1. Global energy demand The increasing global population and the continual rise of modern equipment have made the demand for energy a crucial prerequisite for the success of nations. As a result, there has been a substantial surge in the global demand for energy. This trend has been influenced by the manufacturing of substantial electrical equipment, the expansion of industrial facilities in metropolitan regions, and the increasing population. The climate’s vulnerability to non-renewable energy sources has become more evident due to the simultaneous expansion of modern devices and the increasing worldwide population. Consequently, there is an increasing need for sustainable energy sources such as hydroelectric, geothermal, wind, and solar power. Several nations aspire to achieve complete reliance on renewable energy sources by the year 2050. Nevertheless, the utilization of renewable energy sources is merely a component of the overall solution. Furthermore, it is imperative to ensure efficient harnessing of this energy resource by means of meticulous strategizing and equitable allocation. Grid systems aim to pro- 35 42 Chapter 3. Energy Systems and Forecasting 3.2. Power manufacturing There are several aspects that influence the regulation of energy generation in electric power facilities. The installation and development of electric power stations are influenced by variables such as geolocations and the availability of fuel resources required for their operation. For example, factors such as accessibility to fuel sources, proximity to electric transmission lines and distribution stations, proximity to customers, and public transit networks. The energy consumption in Iraq has risen due to population growth, economic expansion, and a substantial surge in the use of electrical and electronic gadgets by customers [75]. As a result, there is an increase in economic development initiatives that aim to progressively improve the individual level of Iraqis ([133]. There are several obstacles, such as increasing workload or unexpected delays in project completion, that need to be addressed in order to effectively manage unanticipated difficulties or power outages. Moreover, as stated in the Ministry of Electricity of Iraq’s annual statistics report, the current need for electricity production to meet demand needs is 27,000 MVA. Iraq’s current electricity supply is approximately between 15,000 to 17,000 MVA, well below the necessary threshold (Iraqi Ministry of Electricity, 2018, 2019). The deficit in electrical supply is offset by procuring electricity via transmission lines from Iran and Turkey [12] The International Energy Agency (IEA) predicts that electricity consumption in Iraq would increase from 75 terawatt-hours (TWh) in 2018 to 150 TWh by 2030. Figure 3.2 displays the yearly average generation, maximum demand, actual maximum generation, and anticipated maximum generation from 2003 to 2018. Calculations are expressed in megawatts (MW) [85]. Figure 3.2: Generation Portfolio (2003 – 2018) [85] 3.3. Energy resources 43 3.3. Energy resources Iraq ranks among the world’s top countries in terms of verified deposits of fossil fuels. The nation has the fifth largest confirmed reserves of crude oil, accounting for 8.52% of the global total. The worldwide proportion of natural gas reserves is 1.82%, making it the 12th largest in the world. When comparing Iraq’s fossil fuel resources to other global leaders, it is evident that Iraq has much lower oil reserves compared to Venezuela and Saudi Arabia, with a difference of almost two times. Additionally, Iraq’s gas reserves are more than twelve times lower than those of Russia. Iraq’s population constitutes around 0.55% of the global population [58]. The G.P.Ss, which constitute the first stage of the IEPG and are tasked with power generation, rely on two primary resources for their operation (Iraqi Ministry of Electricity in 2018 and 2019) [10]: 3.3.1. Fossil energy Iraq has vast deposits of fossil resources, namely oil and natural gas, which are crucial to its economy and energy production. These resources account for 81% of the country’s electrical generation capacity (Iraqi Ministry of Electricity, 2018, 2019). Iraq holds some of the world’s largest oil reserves, producing about 4.5 million barrels of oil per day, with oil exports comprising over 90% of government revenue. Despite the significance of these resources, the sector faces several challenges. Political instability, aging infrastructure, environmental degradation, and heavy reliance on oil pose substantial obstacles. However, there are opportunities for investment and development, particularly through initiatives like the Basra Gas Company, which aims to enhance gas utilization. To ensure long-term energy security and economic stability, Iraq must focus on regulatory reforms and diversify its energy portfolio by incorporating renewable energy sources. This diversification is essential to mitigate the risks associated with overdependence on fossil fuels and to promote sustainable economic growth. 3.3.2. Renewable energy Renewable energy (RE) refers to a range of alternative and sustainable energy sources that come from natural resources. The sources encompass solar, wind, tidal, hydroelectric, biomass, geothermal, hydrogen, and fuel cell energy. Renewable energy is typically considered an unlimited resource for human use, setting it apart from traditional energy sources that rely on fossil fuels. In Iraq, the main sources of renewable energy include solar, wind, and hydroelectric power. Hydroelectric power is mostly generated from the Tigris and Euphrates rivers using dams [1]. In Iraq, renewable energy accounts for 44 Chapter 3. Energy Systems and Forecasting around 19% of the whole power generating capacity. Furthermore, the stored energy enables the efficient transfer of electricity, especially in situations when conventional means of conduction may be interrupted or unsuitable. In Iraq, the main source of power is hydroelectricity generated from dams [11][8]. To address the increasing need for power in Iraq, it is necessary to use renewable energy sources including solar, wind, and biomass [91]. The effectiveness of these renewable sources is influenced by changes over time and weather conditions, which have a substantial impact on both the production and consumption of electricity [177] [100]. 3.3.3. Solar systems Given the presence of fossil fuels in Iraq, it is imperative to reassess their use in order to safeguard the environment and prevent pollution. Hence, the advancement of technology must be given utmost priority. Solar energy is a very significant renewable resource due to its limitless and environmentally friendly nature. Furthermore, solar energy is a ubiquitous source of energy that is devoid of pollution, secure, and also ecologically sustainable and amicable. Furthermore, it is not bound by geographical or political constraints and does not need any fuel usage. Iraq is regarded as one of the nations with the most significant level of sun radiation. The climate of Iraq is characterized by scorching temperatures, reaching up to 56 degrees Celsius, during the summer season which spans from May to September [1]. The pace of temperature increase differs across different regions, with the Iraqi desert, which covers over 60% of Iraq’s surface, seeing higher temperatures. Furthermore, their occurrence increases in the middle, western, and southern parts of Iraq, but it decreases as we go towards the north. The sun radiation intensity peaks at 1000 watts per square meter during midday, with an average daily range of 250 to 300 watts per square meter. The value is equal to 6 kilowatt-hours per square meter per day, as stated by [14]. The current state of crude oil production and its associated expenses has generated significant interest in many key concerns. Specifically, the areas of emphasis for our study include carbon dioxide emissions, industrial processes, human productivity in various occupations, and the electric distribution power grid. Moreover, the significance of solar energy is growing as a result of climatological shifts in the occurrence of global warming occurrences. Harnessing solar energy offers a means to mitigate carbon dioxide emissions resulting from the combustion of fossil fuels. In addition to attaining energy self-sufficiency, the surplus electricity is exported to neighboring nations. Photovoltaic cells are the primary and most extensive substitute for solar energy, as shown in Figure 3.3. These solar cells are cost-effective and have short installation timeframes. Furthermore, the design of photovoltaic sta- 3.3. Energy resources 45 tions offers greater flexibility in terms of size, making them easily deployable and storable, as well as transferable to other areas [1] [85]. Figure 3.3: Iraq’s solar power plant [136]. 3.3.4. Wind energy The growing emphasis on wind energy in Iraq’s energy industry demonstrates a rising interest in alternative energy sources to tackle the country’s ongoing electricity shortages and support international renewable energy efforts. Iraq, with its vast desert landscapes and coastal regions, possesses significant opportunities for harnessing wind energy. Wind power provides a sustainable answer to the country’s energy concerns by transforming the energy from wind motion into electricity without releasing any greenhouse emissions. Iraq’s significant oil reserves do not translate into sufficient power supply due to an antiquated and inefficient energy infrastructure. This problem is worsened by the increasing electrical demand resulting from population expansion and industrialization. In light of these concerns, the Iraqi government is investigating wind power as a viable alternative to traditional energy sources. According to research conducted by the Renewable Energy Organization, Iraq has the potential to produce more than 5,000 megawatts of electricity from wind power, which would greatly decrease its reliance on fossil fuels. However, the progress of wind energy faces significant obstacles, such as the requirement for huge financial expenditures and the need to overcome technological and legal constraints. However, the government’s efforts to promote the use of renewable energy, shown by the declaration of a 50-megawatt wind farm project near Najaf in 2019, indicate a favorable transition towards 46 Chapter 3. Energy Systems and Forecasting adopting wind power. The study highlights the capacity of Iraq to address its energy deficiencies and reduce its environmental footprint by focusing on the growth of wind energy. Although there are significant hurdles, Iraq’s dedication to renewable energy projects and the increasing interest from global stakeholders and investors indicate a positive path for the country’s energy future[18][48]. 3.3.5. Hydropower resources The abundant water resources of Iraq, mostly sourced from the Tigris and Euphrates rivers, offer a strong foundation for the development of hydropower, which is a crucial element of the country’s renewable energy plan. The crucial functions of the Mosul, Haditha, Dukan, and other dams exemplify the tremendous impact of hydropower on Iraq’s energy infrastructure, providing a large portion of its renewable electricity. Although Iraq has a favorable infrastructure, it has difficulties such as unpredictable changes in water levels and seasonal fluctuations in water flow. These issues are driven by environmental and geopolitical variables and can have a substantial impact on electricity generation. The integration of GIS for accurate water management and turbine efficiency improvement is essential for improving the predictability and stability of hydropower output. Intensive research conducted on infrastructure such as the Mosul Dam use advanced modeling methods to forecast water inflows and enhance turbine operations, hence increasing energy generation. In the future, Iraq is eager to enhance its hydropower infrastructure by implementing new projects and improving current facilities to increase efficiency and capacity. This strategic expansion aims to not only meet domestic energy needs but also contribute to internationally coordinated sustainable water and energy management practices in line with environmental goals. The future of Iraq’s energy security is closely connected to its capacity to enhance the efficiency and sustainability of its hydropower industry. To improve the sector’s performance, promote economic growth, and achieve environmental sustainability, it is essential to tackle the current issues and take use of new technical advancements. This comprehensive strategy guarantees that Iraq can sustain and enhance its position as a prominent player in renewable energy within the region, promoting a more environmentally friendly and robust energy future [86]. 3.3. Energy resources 47 Figure 3.4: Mosul dam [4]. Figure 3.5: Haditha dam [141]. 48 Chapter 3. Energy Systems and Forecasting 3.4. Power supply and demand (challenges and efforts) The energy sector in Iraq is confronted with substantial obstacles arising from a convergence of geopolitical instability, antiquated infrastructure, and a swiftly escalating demand. The country’s security situation has worsened the mismatch between power supply and demand, leading to a significant decrease in electric power output and causing severe shortages in supply. The energy demand situation in Iraq is further complicated by the introduction of new technologies, faults in strategic planning, mismanagement, and shortfalls in infrastructure. Other reasons that contribute to the energy deficit include insufficient gas supply, reliance on outdated grid infrastructure, vulnerability of power stations and transmission lines to sabotage, and the lack of implementation of advanced energy management technology like smart meters [17]. Iraq has been endeavoring to enhance and modernize its infrastructure in order to synchronize with the current technical breakthroughs in the electrical industry. Notwithstanding these endeavors, the demand for power greatly surpasses the present supply and manufacturing capability. A significant number of power plants, constructed from the mid-1970s to the 1980s, together with a limited number of small gas-fired facilities that have been in service since 2003, predominantly depend on crude oil as their main source of energy. These power plants are supplemented by gas and hydropower plants. Privately owned distributed diesel generators somewhat help bridge the gap between demand and supply. By January 9, 2021, Iraq’s electricity production and imports were between 19 to 21 thousand megawatts. However, the actual demand exceeded 30 thousand megawatts, highlighting the urgent need to almost double the production capacity in order to ensure stable electrical energy levels. This is particularly important considering the projected population growth by 2050. The electrical outages, especially noticeable during extremely high summer temperatures over fifty degrees Celsius, emphasize the magnitude of the supply deficit, which exceeds 10,000 megawatts. The factors that contribute to this scenario include deliberate sabotage of electric power networks and decreased gas emissions. The challenges faced include the supply of resources from Iran for station operations, non-compliance with approved energy quotas by governorates, the growth of informal agricultural and squatter areas which further strain the system, increasing temperatures leading to technical problems, and the outdated transmission and distribution networks. The hierarchical organization of Iraq’s energy system, with the Ministry of Electricity as the central authority responsible for managing all areas such as supply, invoicing, and regulation, has resulted in inefficiencies and 3.5. Power grid in Iraq 49 internal disputes, which have further deteriorated the quality of service. The ministry’s many responsibilities as a policymaker, operator, regulator, and supplier give rise to possible conflicts of interest, which are further complicated by the absence of a clear regulatory framework and inadequate customer involvement. To rectify the electricity supply and demand mismatch in Iraq, a comprehensive strategy is needed, which includes improving infrastructure and implementing Contemporary energy management technology and policy improvements. Improving the regulatory framework, increasing consumer engagement, and promoting international cooperation for technical and infrastructural advancement are essential measures to stabilize Iraq’s energy industry. Moreover, allocating resources towards renewable energy sources and enhancing the efficiency of energy production and delivery might have substantial impacts in surmounting the existing obstacles. In order to achieve a secure and sustainable energy future for Iraq, it is necessary to implement not just technology advancements, but also strategic planning, managerial changes, and a strong dedication to environmental sustainability. 3.5. Power grid in Iraq In recent years, Iraq has witnessed a marked increase in electricity demand, driven by significant urban development. This upsurge is attributed to the country’s population growth, a burgeoning economy, and enhanced consumer appetite for electronic appliances, all of which have cumulatively spurred higher energy consumption [1]. Concurrently, the execution of diverse economic development initiatives has been instrumental in improving the living standards of the Iraqi populace [3]. It is imperative to acknowledge the multifaceted factors that influence the power supply capabilities of power stations. Among these, the accessibility to fuel resources and the geographical positioning of power stations relative to consumers are paramount. Furthermore, the technical specifications and strategic placement of these stations, taking into account variables such as population density and the layout of public transportation networks, are critical in determining the efficiency of power distribution. This interrelation of factors highlights the intricate challenges involved in catering to Iraq’s escalating energy demands [1]. The Iraqi Power Grid (IPG), a complex interconnected system, is designed for the generation, transmission, and distribution of electrical energy. It comprises various types of power stations,(Iraqi Ministry of Electricity, 2018, 2019; [75]) as outlined below: Generation Power Stations: (also called Power Plants) are consi- 50 Chapter 3. Energy Systems and Forecasting dered the main source of power. Ultra-High Voltage Stations_400: (Super Grid Stations) which reduce Ultra-High Voltages_400 kv to High Voltages_132 kv. High Voltage Substations_132: (Substations) which reduce High Voltages_132 kv to both Medium Voltages_33 kv and to Low Voltages_11 kv respectively. These medium and low voltages supply the distribution power stations and the consumers with electrical power directly. Moreover, these G.P.Ss are divided into several types of power stations depending on their sources (Iraqi Ministry of Electricity, 2018, 2019) [128]: Thermal Power Stations (Th.P.S): They work on fossil fuels (Pure Oil). Gas Power Stations (G.P.S): Which work on fossil fuels (Natural Gas). Diesel Power Stations (D.P.S): These are based on Gas Oil. Hydro Power Stations (H.P.S): They depend on Renewable Energy (Hydro Energy from Dam) from the two rivers (Tigris and Euphrates). All of these types of power plants are dispersed over Iraq. Figure 3.6 displays the official map of Iraq, including all governorates. Figure 3.6: Iraq Map 3.5. Power grid in Iraq 51 The primary purpose of a power grid is to deliver electricity from the point of generation to end-users efficiently and reliably. The power grid operates on the principle of balancing the supply and demand of electricity as much as possible. It includes various components such as, Generation Stations (GS) or Power plants where electricity is produced and these can be thermal, hydroelectric, or renewable energy sources (see Table3.1). The second is Ultra High-voltage substations (UHV-400) that transport electricity over long distances from power plants to substations. The third is High Voltage Substations (HV-132) that transform voltage levels for efficient transmission and distribution. They also play a key role in switching and controlling the flow of electricity in Iraq. Table 3.2) presents the distributions of stations alongside the other details. Finally, Distribution System which contains the network of lines and equipment that delivers electricity from substations to homes, businesses, and other end-users. Power grids can be regional or national, and in some cases, they are interconnected across borders to facilitate the exchange of electricity between different regions or countries. The reliability and efficiency of a power grid are crucial for ensuring a stable power supply to meet the demands of consumers[61](Iraqi Ministry of Electricity, 2022). Table 3.1: Current types of power stations (GS) used in the Iraqi power grid network. Station Type # of Stations # of Units Designed Capacity (MVA) # of Active Units Production Capacity (MVA) % of Energy Production Thermal Power Stations 8 32 10554 23 4821 24.723 Gaz Power Stations 39 231 26378 157 12989 66.61 Diesel Power Stations 6 51 1309 38 310 1.589 Hydro Power Station 9 33 2944 26 1380 7.076 Table 3.2: Distributions of stations in Iraq and their details. City # of G.P.S # of U.H.V.S # of H.V.S Production Capacity % of Energy Production Duhuk 1 0 5 600 3.076 Mosul 5 1 22 1184 6.071 Salahuldeen 3 0 13 882 4.52 Erbil 2 1 6 985 5.05 Sulaymania 4 0 5 1550 7.94 Kirkuk 4 1 9 831 4.26 Baghdad 9 8 60 2915 14.94 Diyala 2 1 7 501 2.56 Kut 1 1 10 2061 10.56 Anbar 2 1 14 346 1.77 Najaf 2 0 5 543 2.78 Babylon 4 1 12 946 4.85 Kerbela 2 0 5 160 0.82 Qadisiya 1 1 4 330 1.69 Samawa 1 1 5 28 0.14 Nasiriya 2 0 6 779 3.99 Ammara 2 1 3 506 2.59 Basrah 11 2 41 4353 22.32 58 Chapter 3. Energy Systems and Forecasting used to extract relevant information from a cell state [114]. Figure 3.9: LSTM structure [114]. Temporal Convolutional Network. The Temporal Convolutional Network (TCN) places more emphasis on temporal series. The TCN is a time-series processing algorithm developed by Bai et al. in 2018 to address the challenge of extracting long-term time-series information. It combines causal convolution, dilated convolution, and residual blocks. The TCN requires low memory for training due to its shared convolutional filters and can process long input sequences through parallel convolutions, making it a more stable training scheme [62] [27]. Figure 3.10: TCN structure [27]. Multi-layer Perceptron. The Multilayer Perceptron (MLP) is one of the most popular neural networks used to train deep learning models. In this network, the input is presented to the network with the desired output, and the weights are adjusted so that the network attempts to produce the desired output. The MLP consists of three layers: the input layer, which contains the input neurons that feed information to the hidden layer; the hidden layer, which performs calculations based on the input data and forwards the output to the output layer; and the output layer, which represents the model results. The number of 3.8. Time series forecasting models 59 hidden layers determines the depth of the network, and it is the reason why an MLP network with more hidden layers is considered a deep learning model [184]. Figure 3.11: The MLP neural network typically consists of three types of layers: the input layer, the hidden layer(s), and the output layer [5]. 3.8.2. Machine learning models Machine learning models play a crucial role in the advancement of artificial intelligence. They allow systems to train and enhance themselves independently by employing algorithms in supervised, unsupervised, and reinforcement learning frameworks. Supervised learning algorithms, such as linear regression and support vector machines, acquire knowledge from data that has been labeled in order to carry out tasks such as classification and regression. Unsupervised learning, including techniques like as K-means clustering and principal component analysis, is centered around the detection of patterns in data that lacks labels. These models rely on strong mathematical and statistical approaches, including optimization methods like gradient descent, which are essential for achieving accurate and efficient training. The extensive utilization of machine learning in many industries such as healthcare, banking, and automotive has fundamentally transformed predictive analytics and automation, significantly improving the scalability and accuracy of technological solutions. With the continuous evolution of the field, fueled by the growing availability of datasets and processing capacity, machine learning is poised to have a greater impact on both the theoretical and practical aspects of several disciplines [124]. Linear Regression. Regression analysis is a statistical approach that allows us to determine the strength of the relationship between one or more variables. It can therefore help us predict unknown values based on these relationships. simple linear regression involves using one independent variable to model a linear relationship with a dependent variable. multiple linear regression, on the other hand, involves using multiple independent variables to predict the dependent variable 60 Chapter 3. Energy Systems and Forecasting [83]. Although Linear Regression is not inherently suited for modeling non-linear time series data, in this context, some justifications can be found for using it as a prediction model. Linear regression models are simple and easy to interpret and can be a good starting point for modeling time-series data, providing a baseline for understanding the data’s structure. After evaluating the performance of the model, linear regression is considered an appropriate choice for the data under study, as shown by the experimental results [181]. Figure 3.12: Structure of simple and multiple Linear Regression [181]. XGBoost. XGBoost is a popular gradient boosting algorithm used for machine learning tasks. It was developed by Tianqi Chen [44] as an improvement on the GBM algorithm, using a more regularized model to prevent overfitting. XGBoost is known for its efficiency, flexibility, and portability and has been shown to outperform other algorithms in tasks such as classification, regression, and ranking. The algorithm combines multiple weak learners to create a strong learner, with each weak learner trained on a subset of the data. The algorithm works by training decision trees and combining their predictions to make a final prediction [108]. 3.8. Time series forecasting models 61 Figure 3.13: Simplified structure of XGBoost [186]. Random Forest. Random forest models are a popular type of nonparametric machine learning model used for both classification and regression tasks. They belong to the ensemble method category, specifically bagging methods. Ensemble methods use a group of weak learners to create a stronger and more accurate model. Random forests are a collection of many decision trees that are known to be prone to overfitting. By combining multiple trees, random forest models are able to mitigate this issue and provide a more flexible and powerful model with lower variance. This allows larger and more predictive trees to grow, resulting in better performance in both training and unseen data. Additionally, random forest models retain the simplicity and interpretability of decision trees [123]. Figure 3.14: Simplified structure of random forest [186] . 62 Chapter 3. Energy Systems and Forecasting 3.9. Clustering models Clustering is an important unsupervised data mining technique that is used to uncover and reveal underlying patterns in datasets. The primary objective of this approach is to classify data instances, or objects, in a dataset into a predefined number of clusters, ensuring that these clusters are as similar as feasible. The fundamental concept of clustering is to maximize the similarity between things inside each cluster, while minimizing the similarity between objects in different groups. This contrast creates a setting where the inherent similarities and contrasts among the data points are clearly emphasized, facilitating a more profound comprehension of the structure of the information. The clustering process has a value that goes beyond just organizing data; it is an essential tool for doing exploratory data analysis. This is especially accurate in situations when the fundamental patterns, structures, and connections within the data are unfamiliar and there are no predetermined categories to direct the research. The essential distinction between clustering and classification is found here. The former excels in discovering concealed patterns without explicit labels, whereas the latter relies on distinct labels and aims to assign new data points to these pre-established categories [7]. Within the scope of this work, the use of clustering approach is crucial for analyzing and comprehending intricate facts, specifically those related to temperature, power supply, and power demand. The study employs three independent clustering algorithms, each with its own benefits and viewpoints, to conduct a comprehensive analysis of the data. This strategy not only enhances the depth of the analytical insights obtained but also highlights the flexibility and effectiveness of clustering as a data analysis tool, especially in revealing important patterns and relationships within the data that may not be immediately obvious [41]. K-means Algorithm. The K-means algorithm is a popular iterative clustering technique used for partitioning data points into K distinct clusters based on their similarity. It operates by iteratively assigning data points to clusters and adjusting the cluster centroids until convergence is reached. The algorithm requires the specification of the number of clusters, K, at the outset [169]. The K-means algorithm was applied to the preprocessed data with specific considerations for initialization and convergence. To initialize the algorithm, K initial cluster centroids were randomly placed in the feature space. The algorithm then iteratively assigned data points to the nearest cluster centroid based on Euclidean distance. After each assignment, the centroids were updated by computing the mean of the data points within each cluster. This iterative process continued until 3.9. Clustering models 63 convergence, where no significant changes in cluster assignments or centroids were observed [205]. Figure 3.15: K-means Clustering Algorithm [89]. Self-Organized Map (SOM)Algorithm. In this approach, the Self- Organized Map (SOM) algorithm is a powerful unsupervised learning technique widely used for clustering and dimensionality reduction. It operates on the principle of competitive learning, where neurons in a grid-like structure compete to represent input data points. The SOM algorithm possesses two primary advantages: it preserves the topological properties of the input space and is adept at identifying non-linear relationships among data points [127]. The SOM algorithm was applied with careful consideration of the datasets characteristics. The grid size (som_x x som_y) was determined using the square root of the fourth root of the number of data points, ensuring an optimal representation of the underlying patterns. The SOM was then trained using the temperature, power supply, and power demand data. This training process allowed the algorithm to adapt and organize the neurons in response to the input data [172] [55]. Figure 3.16: SOM Clustering Algorithm [151]. 64 Chapter 3. Energy Systems and Forecasting Consensus Clustering. Consensus clustering is a technique employed to enhance the stability and robustness of clustering results. It involves creating an ensemble of clustering solutions by running multiple clustering algorithms with random initializations or parameter settings. This process yields a set of partitions, each representing a different clustering solution [96]. In this study, consensus clustering was utilized to ensure more reliable and consistent clustering outcomes. By aggregating results from multiple runs of both SOM and K-means algorithms, a consensus matrix was constructed to measure the level of agreement among different partitions. This consensus matrix was subsequently used to obtain a final consensus clustering solution, which offered a more stable representation of the underlying cluster structure [90]. This comprehensive clustering methodology enabled a thorough examination of the temperature, power supply, and power demand data, ultimately leading to meaningful insights into the energy dynamics of the cities under study as shown in the table below. Figure 3.17: Consensus Clustering Process [66]. 3.9. Clustering models 65 Table 3.3: Overview and Comparative Study of SOM, K-means, and Consensus Clustering Algorithms Clustering Methodology Technique Description SOM Algorithm - Unsupervised learning. - Preserves topological properties of input space. - Identifies non-linear relationships among data points. - Grid size determined based on dataset characteristics. K-means Algorithm - Iterative clustering technique. - Partition data into K distinct clusters based on similarity. - Initialization with K random cluster centroids. - Iterative task and centroid update until convergence. Consensus Clustering - Enhances stability and robustness of clustering results. - Aggregates results from multiple clustering runs. - Creates ensemble of clustering solutions. - Creates matrix for partition agreement measurement. - Obtains final consensus clustering solution. 66 Chapter 3. Energy Systems and Forecasting 3.10. Complex networks The study of complex networks, involving interconnected nodes and edges, has become prominent in various disciplines. Examples include social networks, the internet, citation networks, neural networks, and power grids. Çomplexityïn these networks can refer to qualitative emergent properties or the quantitative information needed to describe them. The field has roots in mathematical graph theory, with early contributions from Leonhard Euler in 1736 and later advancements by Erdős and Rényi in the 1950s, who developed random network theory. Modern computing advances have revealed that real-world networks are neither fully regular nor entirely random, with notable phenomena such as the small-world and scale-free effects. The work focuses on addressing the challenges faced by the Iraqi Power Grid (IPG) through the application of complex networks theory. It outlines the significant gap between power supply and demand in Iraq and proposes several scenarios to improve the IPG’s performance. These scenarios are visualized using complex networks concepts and analyzed with various network metrics. The work also discusses the methodology for data gathering, network modeling, and the evaluation of scenarios through network metrics like diameter, density, clustering coefficient, average path length, and degree distribution. It concludes with the effectiveness of the proposed models in enhancing the IPG network’s efficiency and outlines future work for further development [142] [185]. The use of this technology has enabled in-depth examinations of power grid networks, such as investigations of cascading failures within the power grid of the United States. These studies have combined CN approaches with accurate load distributions across nodes [118]. Moreover, it has facilitated the analysis of how interrelated dependencies across systems impact the propagation of effects and susceptibility in power networks, such as those in the Southeastern United States [37]. Lately, there has been a growing trend in the literature to use graph theory and mathematical or statistical methods to model power grid networks. The CN technique has shown efficacy in evaluating network performance using statistical and analytical assessments. The power grid network is seen as an intricate system that includes generators, substations, transformers, and large transmission lines [75]. The notion of Scale-Free networks, optimized both numerically and analytically, highlights the adaptability and resilience of these systems in the face of random node losses [39] [46] [147]. In order to guarantee that our conclusions are relevant and applicable to contemporary energy forecasting difficulties, this work also takes into account the most advanced papers, especially from the recent decade. This study endeavors to provide a complete overview of the last literature, despi- 3.10. Complex networks 67 te the constantly changing landscape of energy forecasting, which includes the introduction of new methodologies, data sources, and insights. This study recognizes the inherent restrictions of doing a literature search and aims to provide a comprehensive assessment. However, it emphasizes the need for readers to consider the constraints of the sources used and the timeframe of the work when interpreting the results and conclusions. This article provides an overview of the recognized challenges to validity, emphasizing the meticulous attention dedicated to preserving the integrity and relevance of the research in the fast progressing area of energy studies. This study enhances the knowledge and development of Iraq’s power grid by using CN ideas to simulate the IEPG network. The model assists officials in the Ministry of Electricity by offering practical insights for network growth and failure response techniques. Additionally, it suggests many scenarios to tackle the issues encountered by the present IEPG network. This work highlights the distinctiveness of its strategy in data gathering, network metric analysis, and the creation of solutions to improve grid resilience and stability, by conducting a thorough comparison with similar studies. 3.10.1. Network generation and visualization This section elucidates the process of creating and visualizing the network model for the Iraqi Power Grid (IPG) using complex networks theory and Gephi software. This process is pivotal for analyzing the current state of the IPG and evaluating proposed scenarios for its expansion and improvement. 3.10.1.1. Cumulative dataset for IPG modeling The foundation of network generation lies in the compilation of a comprehensive dataset, which is divided into two primary files. The first file catalogs the nodes (power stations), incorporating essential details such as station ID, name, voltage, location (latitude and longitude), capacity (design and production), type, area, and the number of units per station. The second file details the edges (transmission lines) connecting these stations, with specifics on voltage, weight, capacity, length, and other pertinent attributes. These datasets serve as inputs for Gephi, an open-source network visualization tool, facilitating the intricate visualization and analysis of the IPG network and the scenarios proposed for its enhancement. 3.10.1.2. Network modeling With the dataset prepared, the next step involves modeling the IPG as a graph, where power stations are represented as nodes and transmission lines as edges. This graph is depicted as an undirected, static, and scale-free network, which is instrumental in visualizing and analyzing the network’s 74 Chapter 4. Introduction Table 4.1: Summary of the threat to validate of this work. Strings and Keywords Publishers Databases Energy Forecasting", Elsevier,Web of Science, "Power Forecasting",IEEE,Scopus, .Electricity Forecasting",MDPI,PubMed, .Electricity Prediction",Springer,IEEE Xplore, .Electricity Supply Forecasting",Wiley& Sons,ScienceDirect, .Electricity Supply Prediction",IntechOpen,DOAJ, .Electricity Demand Prediction",Intellect, and JSTOR .Electricity Demand Forecasting",Citeseer, "Machine Learning",UDA, "Deep Learning",PeerJ, Ïraq Energy Sector",MCB UP "Power Demand", and Hindawi and "Power Supply" Chapter 5 Models of predicting power supply and demand The studies conducted on power supply and demand prediction models in Iraq use an extensive study using diverse machine learning and deep learning methodologies. This study employed data from 2019 to 2021 and employed several models including Linear Regression, XGBoost, Random Forest, Long Short-Term Memory (LSTM), Temporal Convolutional Network (TCN), and Multilayer Perceptron (MLP). The models underwent thorough testing over several predicting periods. The empirical data revealed that Linear Regression was the most successful model for demand forecasting, whereas XGBoost exhibited higher performance in forecasting supply. The main objective of this investigation was to enhance energy planning and management techniques in Iraq. This study focuses on the need of precise forecasting in energy policy-making. It aims to address the strategic requirements for overcoming energy difficulties and improving the resilience of energy infrastructure and policy decisions in Iraq. Over the last few decades, researchers have relied on the use of statistical methods to forecast energy demand, energy balance, or energy supply. A commonly used method is the autoregressive integrated moving average (ARIMA), which has been successful in predicting energy demand in stable load situations. However, this method is not always effective in real-world scenarios, where extreme peak loads can occur intermittently [182]. Moreover, with the advent of neural networks, better solutions are now available to researchers. Neural networks can learn hidden patterns from data, which is a significant improvement over purely statistical methods. In fact, neural networks operate similarly to how humans learn: by making a prediction, receiving feedback, and then adjusting their prediction accordingly. A deep neural network that accounts for sequential data is useful for applications that involve time. The literature includes a large number of studies on predicting electri- 75 76 Chapter 5. Models of predicting power supply and demand city supply and demand. Researchers have employed a variety of methods, ranging from statistically based approaches to machine learning and deep learning methods. The choice of a particular method depends on the specific characteristics of the dataset, such as whether it is a time series, has seasonality, or is stationary. Consequently, researchers typically test their dataset before selecting an appropriate algorithm. For example, Dittmer et al [53] forecast the demand for electricity in a rural region in Germany. They first examined their data for seasonality and trends and subsequently used ARIMA and other statistical models to forecast 48 hours ahead. The results showed that the models they employed were suitable for performing the forecasting task and allowed predictions to be made up to 14 days in advance. In another study, Kim et al. [98] developed a hybrid deep learning model using LSTM and CNN for the prediction of power demand. The study used a real-world dataset, and the results indicated that the hybrid model was more accurate in predicting power demand compared to using each model individually. 5.1. Deep learning models In this section, the deep learning models used in the experiments are reviewed, including the Temporal Convolutional Network (TCN), Multi-layer Perceptron (MLP), and Long Short-Term Memory (LSTM). These models were selected for their effectiveness in handling time series data and capturing complex patterns, which are crucial for accurate energy demand and supply forecasting. 5.1.1. Long Short-Term Memory Network The Long Short-Term Memory (LSTM) algorithm is a deep learning method used for prediction purposes that can handle individual data points or a sequence of data points. It has proven to be an effective algorithm that provides accurate predictions based on recent information in the data. LSTM is capable of retaining information for a long period of time to predict, process, and classify time series datasets. It primarily utilizes four neural networks and memory blocks (cells), which store information and control the information flow using three gates produced by a sigmoid function. The Input Gate is used to include useful information; the Forget Gate is used to discard data that are no longer useful; and Output Gate is used to extract relevant information from a cell state [114]. Gates of LSTM(Input Gate, Forget Gate, and Output Gate) The inputs to the LSTM gates consist of the current time step’s input 5.1. Deep learning models 77 and the previous time step’s hidden state, as shown in figure 5.1. Each of the input, forget, and output gates’values are computed by fully connected layers with sigmoid activation functions, resulting in values within the range of (0, 1). Additionally, an input node is required, typically calculated using atanh activation function. The input gate decides the portion of the input node’s value to add to the current memory cell’s internal state. The forget gate determines whether to retain or discard the current memory value. The output gate controls whether the memory cell influences the output at the current time step [197]. Figure 5.1: Computing the input gate, forget gate, and output gate in an LSTM model [197]. Mathematically, suppose there are hhidden units, the batch size is n, and the number of inputs is d. Thus, the input is Xt∈Rn×dand the hidden state of the previous time step is Ht−1∈Rn×h. Correspondingly, the gates at time step tare defined as follows: the input gate is It∈Rn×h, the forget gate is Ft∈Rn×h, and the output gate is Ot∈Rn×h. They are calculated as follows: It=σ(XtWxi +Ht−1Whi +bi), Ft=σ(XtWxf +Ht−1Whf +bf), Ot=σ(XtWxo +Ht−1Who +bo), where Wxi,Wxf ,Wxo ∈Rd×hand Whi,Whf ,Who ∈Rh×hare weight parameters and bi,bf,bo∈R1×hare bias parameters. Note that broadcasting is triggered during the summation. Using sigmoid functions to map the input values to the interval (0, 1). 78 Chapter 5. Models of predicting power supply and demand 5.1.2. Temporal Convolutional Network The Temporal Convolutional Network (TCN) places more emphasis on temporal series. The TCN is a time-series processing algorithm developed by Bai et al. in 2018 [27] to address the challenge of extracting long-term time-series information. It combines causal convolution, dilated convolution, and residual blocks. The TCN requires low memory for training due to its shared convolutional filters and can process long input sequences through parallel convolutions, making it a more stable training scheme [62]. Temporal Convolutional Networks (TCNs) are designed to handle sequence modeling tasks by leveraging a convolutional approach rather than a recurrent one. The key components of a TCN include: 1. Causal Convolutions: Ensures that there is no information leakage from future to past, maintaining the temporal order of the data. 2. Dilated Convolutions: Allows the network to have a large receptive field without a significant increase in the number of parameters or computational cost. The dilation factor increases exponentially with the depth of the network, enabling the model to capture long-range dependencies efficiently. 3. Residual Connections: Utilized to stabilize the training of very deep networks. These connections allow layers to learn modifications to the identity mapping, which helps in training deeper models by mitigating the vanishing gradient problem. 4. 1D Fully-Convolutional Network (FCN) Architecture: Each hidden layer is the same length as the input layer, achieved by zeropadding, which ensures the network can process sequences of varying lengths while maintaining output consistency. The combination of these components allows TCNs to outperform traditional recurrent networks like LSTMs and GRUs on a variety of sequence modeling tasks by providing a robust mechanism to capture long-term dependencies in the data [27]. 5.1.3. Multi-layer Perceptron The Multilayer Perceptron (MLP) is one of the most popular neural networks used to train deep learning models. In this network, the input is presented to the network with the desired output, and the weights are adjusted so that the network attempts to produce the desired output. The MLP consists of three layers: the input layer, which contains the input neurons that feed information to the hidden layer; the hidden layer, which performs calculations based on the input data and forwards the output to the output 5.2. Machine learning models 79 Figure 5.2: Architectural elements in a TCN. (a) A dilated causal convolution with dilation factors d= 1,2,4and filter size k= 3. The receptive field is able to cover all values from the input sequence. (b) TCN residual block. An 1×1convolution is added when residual input and output have different dimensions. (c) An example of residual connection in a TCN. The blue lines are filters in the residual function, and the green lines are identity mappings [27]. layer; and the output layer, which represents the model results. The number of hidden layers determines the depth of the network, and it is the reason why an MLP network with more hidden layers is considered a deep learning model [184]. 5.2. Machine learning models In this section, the machine learning models used in the experiments are reviewed, including Linear Regression, XGBoost, and Random Forest. These models were selected for their diverse approaches to regression and classification tasks, which are essential for understanding different aspects of the data. The focus on these three models provides a comprehensive analysis of their strengths and applicability in energy forecasting. 5.2.1. Linear Regression Regression analysis is a statistical approach that allows us to determine the strength of the relationship between one or more variables. It can therefore help us predict unknown values based on these relationships. simple linear regression involves using one independent variable to model a linear relationship with a dependent variable. multiple linear regression, on the other hand, involves using multiple independent variables to predict the dependent variable [83]. Although Linear Regression is not inherently suited for modeling non-linear time series data, in this context, some justifications can be found for using it as a prediction model. Linear regression models are simple and easy to interpret and can be a good starting point for modeling 80 Chapter 5. Models of predicting power supply and demand time-series data, providing a baseline for understanding the data’s structure. After evaluating the performance of the model, linear regression is considered an appropriate choice for the data under study. 5.2.2. Random Forest Random forest models are a popular type of nonparametric machine learning model used for both classification and regression tasks. They belong to the ensemble method category, specifically bagging methods. Ensemble methods use a group of weak learners to create a stronger and more accurate model. Random forests are a collection of many decision trees that are known to be prone to overfitting. By combining multiple trees, random forest models are able to mitigate this issue and provide a more flexible and powerful model with lower variance. This allows larger and more predictive trees to grow, resulting in better performance in both training and unseen data. Additionally, random forest models retain the simplicity and interpretability of decision trees [123]. 5.2.3. XGBoost XGBoost is a popular gradient boosting algorithm used for machine learning tasks. It was developed by Tianqi Chen [44] as an improvement on the GBM algorithm, using a more regularized model to prevent overfitting. XG- Boost is known for its efficiency, flexibility, and portability and has been shown to outperform other algorithms in tasks such as classification, regression, and ranking. The algorithm combines multiple weak learners to create a strong learner, with each weak learner trained on a subset of the data. The algorithm works by training decision trees and combining their predictions to make a final prediction [108]. 5.3. Related works This section reviews various studies on electricity demand forecasting, focusing on different methodologies and their performance in other contexts. This review provides a foundation for understanding the current state of the art and identifying gaps that our work aims to address. Namrye Son et al. [171] employed deep neural networks (DNN) and long short-term memory (LSTM) models to predict medium- and long-term electricity demand by analyzing data from two organizations with distinct power usage patterns. Their investigation revealed that the DNN model consistently outperformed the LSTM model in all analyzed metrics, including mean absolute error (MAE), root mean square error (RMSE), CVRMSE, mean absolute percentage error (MAPE), and R2. Moreover, the DNN model is more 5.3. Related works 81 efficient in terms of computational time, regardless of the power consumption pattern. This illustrates the resilience and dependability of the DNN in predicting electric power load. The research also highlights a limitation in the incorporation of meteorological data related to seasons, which has the potential to enhance the precision of future predictions. Abeysingha et al. [2] conducted a comparison of machine learning models used to anticipate power consumption in Thailand. The findings indicated that the integration of Artificial Neural Network (ANN) and Deep Belief Network (DBN) yielded the highest level of accuracy in short-term forecasts (1.44% MAPE over a period of one month). Conversely, ANN alone shown superiority in longer-term predictions (1.47% MAPE over a span of one year). SVM models exhibited lower effectiveness, particularly for longer projections. This emphasizes the significance of selecting models based on the length of the prediction and the complexity of the data, with ensemble approaches possibly enhancing accuracy, but their performance may vary. [145] investigated the issue of forecasting short-term electricity demand in Uruguay over the period 2010 to 2019. They employed a variety of models, including linear regression, ridge, KNN, random forest, gradient boosting, MLP, and ExtraTrees, and used benchmarking metrics such as MAE, MAPE, and RMSE. The results indicated that the models mentioned above were suitable for forecasting the hourly power demand. Similarly, Velasquez et al. [183] analyzed the time series of per demand in Brazil for the period 2014 to 2019, using various forecasting approaches. They found that incorporating regression and seasonality with mixing time-series approaches can help reduce forecasting errors. Other researchers aim to test different approaches and determine the most appropriate method for their dataset. Pallonetto et al. [143] recently compared deep neural networks and the Support Vector Machine (SVM) approach. Their results indicated that LSTM provided more accurate forecasting when the load data used in training was sufficient, while SVM performed better when the load was insufficient. The two approaches were applied to one-hour-ahead and one-day-ahead load forecasting. Similarly, Banga et al. [30] compared the power demand forecasting performance of ten models: ARIMA, Prophet, LR, SVM, XGBoost, RF, KNN, RNN, LSTM, and GRU. They evaluated the performance of these models using metrics such as RMSE, MAE, MAPE, and R2. Their findings suggested that at the hourly and daily levels, the Prophet model provided more accurate forecasting compared to the other models. Additionally, several studies have investigated the energy demand and supply of different countries through prediction processes. For example, Raza et al. [148] focused on Pakistan and aimed to create a balance between power demand and supply for economic purposes. They used the Long-Range Energy Alternatives Planning System (LEAP) to perform forecasts, and the 82 Chapter 5. Models of predicting power supply and demand results suggested that Pakistan can generate more power to meet future needs. Similarly, Jaramillo et al. [88] studied the case of Ecuador and used the SARIMA modeling approach for the monthly forecast of the power demand, which proved to be efficient. These studies demonstrate the importance of forecasting in the achievement of sustainable energy systems worldwide. The research paper [196]titled .ASmart Analysis and Visualization of The Power Forecasting in Pakistanütilizes ARIMA and LSTM machine learning models to predict Pakistan’s major energy consumption for the period of 2019 to 2030. The research examines different variables, such as installed capacity and energy consumption by sector, using historical data from the Hydrocarbon Development Institute of Pakistan (HDIP) from 1999 to 2019. The investigation illustrates that machine learning approaches can offer superior accuracy in forecasting compared to classic econometric models. These models frequently struggle to predict the dynamic changes in the energy industry caused by economic and demographic developments. The results strongly recommend using sophisticated predictive models into energy planning to improve the precision of predictions and facilitate strategic decisionmaking in addressing Pakistan’s increasing energy demands. Gerwig et al. [67] provided a comprehensive analysis of the techniques used in short-term load forecasting (STLF) for residential structures. They also determined the most efficient approaches for certain objectives. A systematic literature review was performed, examining 375 publications and classifying them based on a concept matrix. The research found that the use of artificial neural networks (ANN), autoregressive approaches, and their combination (hybrid methods) are efficient in predicting low-frequency (LF) needs in small-scale households (1–1000) and individual users. Moreover, linear regression proves to be efficient in predicting load forecasting for individual users, but Support Vector Regression (SVR) is appropriate for load forecasting requirements in groups consisting of more than 32 households. Integrating clustering approaches with artificial neural networks (ANN) or autoregressive algorithms has the potential to enhance the precision of the predictions. The report proposes the need for more investigation in this field and recommends using publically accessible datasets for the purpose of evaluating and comparing approaches. Pedro Lara-Benítez and colleagues [105] employed Temporal Convolutional Networks (TCN) to estimate two energy-related time series in Spain: national electric consumption and power demand at charging stations for electric vehicles. A comprehensive experimental analysis involving over 1900 model configurations with diverse topologies and parameterizations revealed that TCN models exhibited enhanced forecasting accuracy compared to Long Short-Term Memory (LSTM) networks, which were previously regarded as the state-of-the-art. TCNs reduced the mean absolute percentage error (MAPE) by successfully capturing long-term relationships through dilated causal 5.3. Related works 83 convolutions. Furthermore, the paper highlights the computational efficiency of TCNs, emphasising their suitability for real-time applications that require rapid prediction capabilities. In [102], existing LF techniques were analyzed to determine which technique is best suited to a certain situation or circumstance. Time frame, inputs, outputs, scale, data sample size, error kind, and value were the criteria used to evaluate these different approaches. Regression and multiple regression were the most often employed methods in LTLF. For STLF and VSTLF applications, ML-based approaches such as ANN, SVM, and time series analysis incorporating auto-regressive integrated moving average (ARIMA) and auto-regressive moving average (ARMA) were used. Runge et al. [156] conducted a review of the studies that have used artificial neural networks (ANN) to anticipate building energy consumption. Moreover, Amasyali et al. [22] conducted a comprehensive evaluation of the advanced data-driven methodologies used for building energy load forecasting. Both surveys examined the practical uses, data sets, predictive models, and metrics for measuring performance. Anindita Golder and colleagues [71] investigate the efficacy of various models, including Support Vector Machines (SVM), Multi-Layer Perceptrons (MLP), and Long Short-Term Memory (LSTM) networks, in forecasting Load Demand and PV generation. The statistical data employed includes aggregated load demand from 40 residences in Austin, Texas, as well as PV generation from Australia’s Yulara Plant, which incorporates local weather conditions. The MLP model demonstrated superior performance in load demand projections, while the LSTM model exhibited greater accuracy in PV generation forecasting. The findings demonstrate that these machine learning models are capable of effectively managing and anticipating energy usage and generation. Pengcheng Zhang et al. [201] extended the use of Multi-layer Perceptron (MLP) for short-term rainfall forecasting by incorporating a Principal Component Analysis (PCA) to reduce the dimensionality of input features and employing a greedy algorithm to determine the optimal structure of the MLP. This architecture was evaluated using data from 56 meteorological sites in China, yielding similar prediction accuracy compared to existing models but achieving significantly faster training times. The DRCF model outperformed other machine learning approaches and atmospheric models in both threat score (TS) and root mean square error (RMSE), highlighting its efficiency and accuracy in short-term rainfall forecasting. The given study [157]examines the utilization of machine learning models to predict energy consumption in Astana. The study specifically focuses on two models: Forecaster autoregression and a neural network. The study incorporated air temperature and wind speed into the prediction analysis for Astana’s unique environment, using a comprehensive dataset spanning Chapter 6 Network Modelling 6.1. Methodology In recent years, Iraq has faced significant challenges in its electrical infrastructure, exacerbated by extensive damage and a growing gap between power supply and demand. The Iraqi Power Grid (IPG), an intricate network of power plants, transmission lines, substations, and distribution systems, currently operates under severe strain, with a supply of 17,000 MVA falling short of the 27,000 MVA demand. This study leverages complex network theory to model and enhance the IPG, providing a comprehensive framework for addressing these challenges. Complex network theory, a powerful tool for analyzing and visualizing large-scale systems, is utilized to represent the IPG as an undirected network. In this model, nodes symbolize power plants while edges represent the transmission lines connecting them. This approach allows for a detailed analysis of the network’s structural properties and performance using various network metrics, such as diameter, density, clustering coefficient, average path length, and degree distribution. To address the operational issues and enhance the functionality of the IPG, three scenarios were devised: 1. Scenario B1: Focuses on the addition of new Generation Power Stations (G.P.S) to boost the power supply by 6750 MVA, addressing immediate shortages. 2. Scenario B2: Proposes the expansion of Ultra High Voltage (UHV- 400) networks to improve transmission capacity by 8000 MVA, ensuring better power distribution and stability. 3. Scenario B3: Involves the expansion of High Voltage (HV-132) networks to enhance distribution capabilities by 7434 MVA, supporting long-term growth and reliability. 91 92 Chapter 6. Network Modelling Each scenario was implemented on the original IPG model, creating new configurations aimed at resolving specific operational issues. The performance of these scenarios was thoroughly evaluated and compared with the original network model using complex network metrics. This comparative analysis demonstrated the scenarioséffectiveness in improving network resilience, efficiency, and reliability. The implications of this study are of great interest to the Ministry of Electricity in Iraq, as the findings offer valuable insights for future policy development and practical applications. By employing advanced network modeling techniques, this research provides a strategic roadmap for enhancing Iraq’s national power infrastructure, ensuring a more stable and reliable power supply for the country’s growing needs. 6.2. Graph Theory Graph Theory (GT) is a mathematical field that originated in the eighteenth century and has recently become integral to numerous disciplines, including information technology and computing. A graph comprises elements known as nodes (or vertices) and their connections, referred to as edges (or links) [63]. A graph G(V, E)is defined by a set of vertices Vand edges E, where each edge links two vertices. Typically, vertices represent objects, while edges denote the relationships between these objects [93]. The entire network of interconnections forms the graph’s topology, offering a detailed map of the relationships among nodes and edges [63]. Graphs can be analyzed using various metrics at both the local (node-level) and global (network-level). Node-level metrics include statistics such as node degree, centrality, and distribution, whereas global metrics cover properties like path length, density, and diameter [174]. Graphs are valuable models for addressing problems in diverse fields such as computer science, operations research, and the natural and social sciences. GT has thus developed into a unified mathematical approach encompassing a wide range of analytical techniques and tools [72]. Graphs can be either directed or undirected. In directed graphs, edges have a specific direction, indicating a path from one vertex to another. For instance, an edge from vertex V1to vertex V2suggests a direction from A to B. This is shown in Figure 6.1, where the initial vertex Aconnects to the terminal vertex Bvia a directed edge [72]. 6.2. Graph Theory 93 Figure 6.1: Directed Graph [52] In contrast, undirected graphs have edges that do not indicate any direction, allowing connections between vertices to be bidirectional. Figure 6.2 illustrates this concept, showing vertices 1 and 2 connected in both directions without directional arrows [72]. Figure 6.2: Undirected Graph [52] Weighted graphs assign numerical values to their edges, representing weights. For example, Figure 6.3 shows an edge with a weight of four between vertices Aand B[52]. Figure 6.3: Weighted Graph [52] Graphs can be further categorized based on their structure [174]: 94 Chapter 6. Network Modelling Simple Graph: An undirected graph without self-loops or parallel edges, depicted in Figure 6.4 [42]. Figure 6.4: Simple Graph [159] Multiple Graph: A graph that may have multiple edges between the same nodes, including self-loops, as shown in Figure 6.5 [131]. Figure 6.5: Multiple Graph [126] Complete Graph: A graph where each pair of vertices is connected by a single edge, illustrated in Figure 6.6 [42]. Figure 6.6: Complete Graph [92] 6.3. Related works 95 Key terms and concepts in graph theory include [174]: Graph Diameter: The greatest distance between any two vertices in the graph. Node Degree: The count of edges connected to a vertex. Shortest Path Length: The minimum distance between two vertices. Graph Degree: The total sum of all vertex degrees in the graph, which equals twice the number of edges. Graph theory has been widely adopted to solve various problems and is utilized in multiple fields, including science, physics, chemistry, engineering, transportation, forensics, spectroscopy, and the study of disease transmission [122]. 6.3. Related works This section reviews various studies on network modeling, focusing on different methodologies and their performance in other contexts. This review provides a foundation for understanding the current state of the art and identifying gaps that our work aims to address. Rasha Al-Jarah et al. [10] enhanced the understanding of power grid networks by modeling the Iraqi Power Grid (IPG) network using the theory of Complex Networks. This study addresses the intricate topological interconnections of power grids, which present significant challenges in terms of analysis and visualization. The proposed model represents power stations as nodes and transmission lines as edges, allowing for a comprehensive analysis using network metrics such as node degree, clustering coefficient, path length, closeness, betweenness, density, and diameter. The IPG network was visualized based on the demographic distribution and accurate coordinates of the power stations. This work provides a valuable tool for the Iraqi Ministry of Electricity, offering insights that can aid officials and specialists in understanding, visualizing, and evaluating the performance of the current IPG network. The findings underscore the potential of Complex Networks theory in improving the efficiency and robustness of power grid networks. 96 Chapter 6. Network Modelling Figure 6.7: Visualization of a segment of the IPG network showing power stations and their interconnections [10]. Ngakan Ketut Acwin Dwijendra et al. [57] advanced the design of hybrid renewable energy systems by proposing an off-grid configuration tailored for a residential house in Hamadan city, Iran. This improved system addresses the challenges of integrating renewable energy resources with varying weather conditions and economic constraints. The model optimizes the sizing and installation of resources such as batteries, solar cells, wind turbines, and electrolyser/fuel cells. By implementing a twofold optimization approach focusing on operation cost (OC) and net present cost (NPC), the study enhances traditional hybrid energy systems through the use of HOMER software for optimal resource allocation. Albert et al.[15] developed a model of the North American power grid network using data sourced from the POWER mapping system, an advancement by Platts, the energy information and market services unit of The McGraw- Hill Companies. This model encompasses details about each power station and its transmission lines, with 115 kV for HV-132 and 765 kV for UHV- 400. The network comprises 14,099 nodes, including 1,633 power stations and 2,179 distribution substations, connected by 19,657 edges representing power transmission lines. The degree distribution adheres to a Power-Law P (k), and within the mapping, a continuous hierarchy of high-degree nodes (hubs) influences the system’s topology in these intricate networks. Sun[175] initiated an analysis and comparison of metrics across three power grid networks: North China, Central China, and the Western American 6.3. Related works 97 power grid networks. Notably, the clustering coefficient of the Chinese Central Power Grid showed a mere fourfold increase from that of a random graph with equivalent order and magnitude. Furthermore, authors in[188][202] delved into the statistical characteristics of the aforementioned networks. Rosato et al.[154] conducted an analysis of the topological statistical properties of Ultra High Voltage power grid networks in several European countries. This included the Italian-380 kV, the French-400 kV, and the Spanish- 400 kV power grids. The detailed Italian grid encompasses electric transmission lines ranging from 380 kV down to 120 kV, based on available data. In the upcoming section, Table 9.2 outlines the statistical characteristics, representing the network metrics of these European networks alongside the IPG. The statistical features of the IPG are determined by calculating its network metrics in a manner close to the aforementioned networks. Khan et al. [95] conducted a comprehensive study of LF approaches that use dynamic pricing systems in smart grids (SGs). The researchers examined several pricing models, including real-time pricing (RTP), time of use (ToU), and critical peak pricing (CPP). They organized several LF approaches into mathematical and AI-driven computer models. Wiese et al. [191]present a thorough and detailed examination of the Balmorel energy system model in their research article. They investigated the use of open-source modeling frameworks to create tools that can analyze energy systems. These tools consider different aspects of time and space to assess and improve system performance. The model essentially employs a versatile strategy to effectively handle fluctuating renewable energy and optimize investments, leveraging the combined expertise of several public and private research partnerships. The model showcased its potential by being utilized in energy planning and policy research. Its performance measurements encompassed the seamless incorporation of renewable energy sources and the optimization of system operations. The knowledge was gathered via a comprehensive record of model creation and application, enhanced by various contributions that encompass scenario building and extension skills. Zhou et al. [203]presented an innovative architecture that utilizes Digital Twin (DT) technology to do real-time online analysis of electricity grids. Their research is around creating an Online Analysis Digital Twin (OADT) that use sophisticated computational methods to efficiently handle the intricacies of extensive power networks. The research showcases the utilization of OADT to enhance system architecture by drastically decreasing the response time of online studies from 10 minutes to less than 60 seconds. The technique utilizes high-fidelity simulations, parallel computing, and machine learning to effectively optimize and forecast grid behaviors. These functionalities are based on a comprehensive virtual model that is constantly updated with realtime data from the physical grid, improving decision-making and operational efficiency. Real-world use of this technology has demonstrated encouraging 98 Chapter 6. Network Modelling outcomes in improving grid stability and operational responsiveness. Asensio et al. [24] introduce an advanced model for enhancing the efficiency of the cooling system in a polymer electrolyte membrane fuel cell (PEMFC)-based combined heat and power (CHP) system. Their study leverages an artificial neural network in conjunction with a three-dimensional lookup table to update data. Additionally, they apply dynamic Hermite functions to interpolate this data. This model precisely forecasts coolant outlet temperatures and hydrogen consumption rates by considering different operating factors. The model greatly improves the creation of novel optimization algorithms for real-time applications, guaranteeing effective energy management in PEMFC systems. The simulation and experimental evaluations highlight the model’s ability to successfully manage complicated and nonlinear data, leading to gains in energy efficiency and cost savings for CHP systems. Panteli et al. [144] conducted a thorough examination of the ability of power systems to withstand extreme weather events by using fragility modeling and probabilistic impact assessment approaches. Their research highlights the creation of robust models for transmission networks, with a specific focus on the consequences of intense windstorms. The study presents an innovative approach to analyze the resilience of power transmission networks to windstorms. This approach combines sequential Monte Carlo simulation with optimum power flow to quantify the spatial and temporal effects of windstorms on the networks. This methodology enables a comprehensive examination of different resilience improvement techniques and the assessment of transmission component resilience using the resilience accomplishment worth index. The results indicate that the suggested models and approaches are successful in enhancing the durability and promptness of power systems in the face of severe weather conditions. This eventually assists in the strategic planning and reinforcement of power infrastructure resilience. Liu et al. [113]present a comprehensive analysis of power system restoration methods, with a specific emphasis on advancements made from 2006 to 2016. The study explores the progress made in black-start capabilities, network reconfiguration, and load restoration, with a focus on the incorporation of variable renewable energy sources and smart grid technology. The report emphasizes notable advancements in automating restoration procedures to improve the resilience of systems against extensive outages. The text explores approaches like as artificial intelligence and wide area measuring systems (WAMS), which are essential for making dynamic decisions during restoration. The assessment also discusses the difficulties presented by cyber-attacks on power system operations, highlighting their capacity to complicate restoration endeavors. In summary, the study emphasizes the continuous requirement for research in the crucial field of power engineering, proposing potential avenues for future endeavors to attain power systems 6.3. Related works 99 that are more dependable and capable of self-repair. The paper [104] offered a research that compared several strategies for predicting power usage and discussed their potential applications. This research determined that a power consumption prediction architecture is used for metering, communication, supervisory control, and data acquisition (SCADA) monitoring. Additionally, a database is employed to store this information for various smart applications, including the mining sector, smart grid, smart cities, and industry. In [78], a comprehensive overview of probabilistic electric LF was presented, which included the main approaches and evaluation procedures. This study offered valuable insights and novel information for academics and practitioners to enhance and advance the area of LF. In their study, Nespoli et al. [135] conducted a comparison of several PLF methodologies to forecast the load demand of secondary substations and cabinets in a distributed low-voltage grid. The methodologies were analyzed using standard Key Performance Indicators (KPIs) for deterministic and probabilistic predictions. In addition, they assessed the efficacy of several hierarchical methods to enhance the performance of predictors at the lower level. The literature emphasizes that complex networks serve as a strong tool for modeling power grids due to their effective visualization and analysis methods. 106 Chapter 7. Clustering Techniques Guiqing Zhang et al. [199] proposed a k-means clustering-based method for electrical equipment identification in smart building applications. They collected real-time and historical data of electrical parameters from a Building Internet of Things (BIoT) platform using smart sockets. The method involved two stages of k-means clustering: preliminary clustering using total harmonic current distortion (THDi) data to separate equipment into clusters based on electrical characteristics, and later clustering using an improved kmeans algorithm with a weighted Euclidean distance measure to refine the clusters. The identification model was constructed by selecting cluster centroid vectors and distance thresholds. This approach was validated through applications to a BIoT system, demonstrating high accuracy in identifying typical electrical equipment. The method effectively reduces calculation complexity and enables real-time equipment identification, enhancing equipment management in smart buildings. Seyyed Mohammad et al. [68] demonstrated that the use of a Self-Organizing Map (SOM)-based clustering approach for adaptive protection in microgrids resulted in a notable improvement in overcurrent relay coordination. The proposed solution effectively addressed the challenges posed by the dynamic and diverse operational settings of microgrids, encompassing grid-connected and islanded modes. The approach optimized relay settings by grouping comparable miscoordinated relay pairs, which not only decreased miscoordination intervals but also increased relay operating times. This methodology demonstrated superior performance in comparison to standard methods with regard to relay performance and protection system responsiveness. This evidence suggests that the methodology has the potential for use in complex power grid systems. Miraftabzadeh et al. [129] investigated the application of K-means and other clustering techniques in contemporary power systems, with a specific emphasis on improving the efficiency and reliability of these systems in the face of growing complexity and data volume. The paper conducted a detailed assessment of the uses of these algorithms, emphasizing their incorporation into tasks such as load forecasting, fault detection, and power quality monitoring. Clustering algorithms have proven essential in managing large datasets and automating decision-making processes in power systems. The research highlights a significant and rapid growth in the use of these approaches, indicating their rising importance in updating power infrastructure to accommodate dynamic and dispersed energy supplies. Bañales et al. [29] investigated a new approach that utilizes time series clustering of smart meters for demand response applications in areas with a significant presence of renewable energy. The study devised a two-stage kmedoids clustering algorithm to examine normalized load-shape time series, which are divided into 48 time points during the day. This strategy successfully decreased the time complexity by first grouping individual customers 7.3. Consensus Clustering 107 and then clustering customer representatives in an aggregate manner. The approach utilized simplified time series representations to improve computing performance. The results showed that customer segmentation based on daily energy consumption patterns was successful, highlighting the model’s capacity to enhance demand response techniques by matching consumer energy usage with renewable energy availability. The validation of this strategy demonstrated favorable outcomes in both internal and external measurements, confirming the suitability of the methodology in real-world scenarios. Rhodes et al. [152] studied an extensive investigation of the demand for energy in residential areas. They analyzed the differences that occur throughout different seasons and using clustering techniques to gain a deeper understanding. The study using k-means clustering to identify unique clusters of home power consumption patterns throughout different seasons. The analysis revealed substantial variations in connection to wholesale electricity market pricing. The regression analysis established correlations between these patterns and demographic and behavioral data obtained from surveys conducted with homeowners. The key findings revealed that telecommuting has a significant effect on the rise in power consumption during periods of high prices. This has important implications for the implementation of demand response systems and the management of utilities. This study highlights the intricate nature of home power usage and provides a scientific framework for creating focused energy management and policy efforts. Bilal Naji Alhasnawi et al. [19] proposed a demand management method for smart microgrids based on a consensus algorithm and coalition game theory. This strategy markedly enhances the distribution of surplus energy among customers by leveraging the Shapley value to address uneven power distribution, while also enhancing data management by incorporating fog computing. Simulations conducted in MATLAB evaluated the approach under three scenarios, demonstrating notable savings in energy expenditures and waste, as well as enhanced control of power imbalances. The strategy not only improved energy distribution and decreased costs, but it also demonstrated the practical benefits of using advanced computational approaches in smart microgrid management. Nowotarski et al. [139] presented a thorough examination of energy price forecasting (EPF) together with the essential instructions for accurately using methodologies, measurements, and tests. It is seen as a continuation and modernization of [190]. The examined subjects are applicable for probabilistic energy forecasting, including VSTLF for SM applications and wind and solar power forecasting. The study conducted by De Jaeger et al. [50] effectively proves that their approach of grouping buildings can accurately forecast both yearly and peak urban energy requirements. This is achieved by just considering local building attributes, rather than relying on actual energy consumption data. This 108 Chapter 7. Clustering Techniques method greatly decreases the processing demands usually linked to intricate simulations conducted at the building level. The technique enhances the efficiency and reduces resource consumption of energy modeling for urban districts by classifying buildings into representative archetypes based on factors such as footprint and construction time. This technique not only helps optimize urban energy systems but also improves the integration of renewable energy resources. It provides a strong tool for urban planners and engineers to promote energy efficiency and system design in urban areas. In the literature review, various studies on clustering techniques in energy systems are summarized in Table 7.1. The table highlights the models, datasets, limitations, and advantages of each study, showcasing the range of approaches used to enhance energy management and forecasting. Key studies include the use of Two-level Clustering (SOM) for residential power usage by Rinku Chavda et al., and K-means Clustering in power systems data by Miraftabzadeh et al. This summary illustrates the diversity of methodologies and emphasizes the importance of model selection based on specific data and goals. Table 7.1: A summary of the literature. Study Model Dataset Limitations Advantages Rinku Chavda et al. [43] Two-level Clustering (SOM) 6,445 dwellings dataset Limited to residential settings, does not consider commercial or industrial use Effectively segments power usage patterns, valuable for targeted energy management Seyyed Mohammad et al. [68] SOM-based Clustering Microgrid relay data High computational cost, complex implementation Improves overcurrent relay coordination, enhances protection system responsiveness Miraftabzadeh et al. [129] K-means Clustering Contemporary power systems data Requires large datasets for accuracy Essential for load forecasting, fault detection, and power quality monitoring Bañales et al. [29] Two-stage k-medoids Clustering Smart meter time series data Initial setup complexity Reduces time complexity, enhances demand response by matching energy usage with renewable availability Rhodes et al. [152] K-means Clustering Residential energy demand data Seasonal variations, requires demographic data for correlation Identifies unique consumption patterns, informs demand response and utility management Bilal Naji Alhasnawi et al. [19] Consensus Algorithm, Coalition Game Theory Smart microgrid data High initial setup cost, requires advanced computational resources Enhances energy distribution, reduces costs, leverages fog computing for improved data management Nowotarski et al. [139] Probabilistic EPF Energy price forecasting data Requires detailed historical price data Provides guidelines for VSTLF and wind/solar power forecasting, applicable to SM applications De Jaeger et al. [50] Building Grouping Urban energy data Does not use actual energy consumption data, relies on building attributes Accurately forecasts yearly and peak urban energy requirements, optimizes urban energy systems Part III Proposals Chapter 8 Models of predicting power supply and demand Models of Predicting Power Supply and Demand discusses the use of various machine learning and deep learning models to forecast electricity supply and demand in Iraq. The study utilized a dataset from 2019 to 2021, employing models like Linear Regression, XGBoost, Random Forest, LSTM, TCN, and MLP. The models were evaluated across different forecast horizons. Key findings revealed Linear Regression as consistently top-performing in demand forecasting, while XGBoost excelled in supply forecasting. The analysis aimed to enhance energy planning and management in Iraq, highlighting the importance of accurate energy forecasting for policy-making and addressing energy challenges. 8.1. Methodology This section describes about the data collection process as well as the forecasting models involved in this work. Also, the setup of the experiments in terms of optimizing the models’parameters and the evaluation merics are also explained. 8.1.1. Data Exploration The data used in this work was officially collected from the Department of Operations and Control of the Ministry of Electricity in Baghdad, Iraq, for the period 2019 to 2021. The data consisted of hourly time-series data on the supply and demand of 15 provinces in Iraq. The collection process was strictly regulated due to governmental procedures in Iraq, which took approximately 4 months to complete. Then it was preprocessed and cleaned to address missing values and outliers. In total, the dataset consisted of 26,352 rows and 15 columns, each corresponding to a different province. 111 112 Chapter 8. Models of predicting power supply and demand An analysis of raw data to assess the energy performance of the country is presented. Furthermore, we delve into climatological seasonal patterns, national holidays, Ramadan, and Sha’ban periods, all of which play significant roles in shaping energy dynamics within the region of Iraq. The dataset encompasses demand and production data measured in megawatts (MW), each structured with a time column spanning from 1:00 on 01-01-2019 to 24:00 on 02-01-2022, alongside columns representing the demand or production values for each of the sixteen provinces. The initial step involves aggregating both variables to ascertain the country total consumption and production, obtaining thus a unified dataset containing hourly demand and production data. Figure 8.1: Power demand and supply in Iraq for 2019, 2020, and 2021. Upon initial examination, figure 8.1 reveals that the country struggles to meet its energy demands, with instances where demand nearly doubles its production capacity. It is understood that the deficit in energy is compensated by importing from neighboring countries. Furthermore, analysis indicates that summer months exhibit the highest energy demands, likely due to the country geographical location. Preceding these months, a decline in demand is observed, prompting further investigation into its underlying causes. While the exact reasons remain to be explored, initial observations suggest a correlation between specific periods within the country and peak demand. Being a country with a geography that marks the different seasons, three main types of regional climates can be identified: Mediterranean/Humid climate: this climatic zone is most common in the northern regions of Iraq and is characterized by mild and humid winters and hot and dry summers. Rainfall tends to be more abundant compared to other regions of the country. Semi-Desert Climate: This type of climate is found in transitional areas between the more humid and arid zones. In these regions, rainfall is scarce and temperatures can be extremely high in summer, while winters are generally milder. 8.1. Methodology 113 Desert climate: Predominant in southern and central Iraq, this climate is characterized by extremely high temperatures in the summer and very dry conditions throughout the year. Rainfall is scarce and, when it does occur, it is usually torrential. From these regional climates, a new feature has been added to the dataset, where the season has been set to Winter for the months from December to March, Spring for April and May, Summer for the months from June to September, and Fall for October and November. Figure 8.2: Demand vs supply during the different seasons. Figure 8.2 illustrates the average demand and production in each season, where it can be seen that the summer season amplifies both. Furthermore, it can also be derived that the disparity between these two variables is more pronounced during summer compared to autumn and spring, where the difference is less noticeable. This variation in seasonal production levels may be due to climatic conditions and their impact in various energy production industries. For instance, solar energy sources experience heightened production during summer due to increased sunlight hours, whereas the opposite occurs in winter. Similarly, hydroelectric power plants witness augmented production during the rainy seasons. Next, we will examine national holiday occurrences to determine if they align with days of significant demand surge. As it can be seen in figure 8.3, demand tends to decrease on holidays, indicating their importance, such as January 1, when many factories halt operations. Days like summer holidays exhibit lower demand, except for July 14, commemorating the great revolution, which stands out as a day with notably increased demand and production during holidays, reflecting its national significance and widespread celebration. 114 Chapter 8. Models of predicting power supply and demand Figure 8.3: Demand vs supply on national public holidays. Furthermore, over the years, these marked holidays witness a rise in demand while production either remains steady or experiences slight growth, highlighting a disparity in the pace of demand and production increase. This analysis underscores the significance and activity levels of the country on these celebrated days. Finally, we have carried out an analysis of the Ramadan and Sha‘ban periods of the years available in the dataset. During these months, people’s daily routines undergo significant changes due to fasting from dawn to dusk. This leads to an increase in nocturnal activity, and thus in the demand for energy. In addition, to conserve energy during the fasting hours, many people choose to stay indoors and in a warm climate, which often translates into increased use of air conditioning. Business hours also adapt, extending late into the night to meet the needs of consumers. Finally, community meetings and prayers increase in frequency, requiring that places for these activities remain open longer than usual. Figure 8.4: Demand vs supply during Ramadan and Sha’ban periods. 8.1. Methodology 115 From figure 8.4 it can be derived that the month of Ramadan significantly impacts the country’s energy consumption. On the other hand, Sha’ban, while serving as an introduction to Ramadan, seems to be of lower importance in terms of energy consumption. These periods really impact in terms of energy in the country, and they will constitute a very good feature for the training of a prediction neural network. 8.1.2. Time series forecasting models One of the main considerations in the analysis of time series data is the examination of their inherent seasonality [180]. Upon conducting a rigorous Dickey-Fuller analysis on the dataset (see Figures 8.5 and 8.6), it was evident that both demand and production exhibited non-stationary behaviors. Consequently, in light of this assessment, the decision was made to employ forecasting models that are robust to the absence of stationarity in the series. This judicious choice not only enhances the robustness of the analysis, but also confers adaptability by bypassing the strict stipulation that the data must conform to stationarity, a conventional prerequisite in many established statistical models. Figure 8.5: Dickey-Fuller Test for Demand. Figure 8.6: Dickey-Fuller Test for Supply. 122 Chapter 8. Models of predicting power supply and demand Table 8.4: The best parametrization for each model and dataset. MODELS DEMAND MODEL DESCRIPTION SUPPLY MODEL DESCRIPTION LSTM Recurrent Layers = 1 Units = 128 Return sequence = False Recurrent dropout = 0 Dense dropout = 0.2 Dense layers = [32, 64, 128] Recurrent Layers = 1 Units = 64 Return sequence = False Recurrent dropout = 0 Dense dropout = 0.2 Dense layers = [32, 64, 128, 64, 32] TCN Number of stacks = 3 Number of filters = 64 Dilations = [1, 2, 4, 8] Kernel size = 3 Return sequences = False TCN dropout = 0.2 Dense dropout = 0.2 Dense layers = [64, 32] Number of stacks = 3 Number of filters = 64 Dilations = [1, 2, 4, 8] Kernel size = 6 Return sequences = True TCN dropout = 0 Dense dropout = 0 Dense layers = [32, 64] MLP Hidden layers = [32] Hidden layers = [32, 64] Fit intercept = False Fit intercept = False LR Normalize = True Normalize = True Positive = False Positive = False Number of estimators = 300 Number of estimators = 300 Max depth = 10 Max depth = 10 RF Min samples split = 4 Min samples split = 2 Min samples leaf = 1 Min samples leaf = 3 Booster = gbtree Booster = gbtree Number of estimators = 300 Number of estimators = 300 XGB Min child weight = 10 Min child weight = 10 Subsample = 0.8 Subsample = 0.8 Colsample bytree = 0.8 Colsample bytree = 0.5 Max depth = 6 Max depth = 6 8.1. Methodology 123 In other words, the optimization of the model parameters was performed through a systematic grid search approach. Specifically, a range of hyperparameter values was predefined for each forecasting model. These parameter combinations were systematically tested on demand and supply datasets. Given the extensive number of prediction horizons and forecast horizons employed in this study, a preliminary phase of parameter optimization was carried out with fixed values for past history (PH) and forecast horizon (FH). Specifically, the values chosen were FH = 24 and PH = 72. Subsequently, after identifying the optimal parameters for this specific FH and PH configuration, they were applied to calculate performance metrics for the remaining forecast horizons and the past history. The performance of each model with different parameter combinations was evaluated using various metrics, including the mean absolute error (MAE), the mean square error (MSE), the root mean square error (RMSE), the weighted absolute percentage error (WAPE) and the mean absolute percentage error (MAPE). As explained above, to enhance the clarity of this procedure, it has been determined that a figure illustrating the process is included in the section Experimental Setup. This can be seen in the figure 8.7. Using different normalization methods in the study serves several purposes: 1. It enhances the robustness of the results by accounting for variations in data characteristics. 2. Enables comparisons to identify the most effective normalization technique. 3. Assesses the generalization of forecasting models to different data representations and aids in data exploration by revealing specific data patterns. 8.1.6. Evaluation metrics The purpose of this study is to evaluate and compare the performance of previous models under different scenarios. To achieve this, five widely recongized metrics from the forecasting literature [45, 36] were selected: mean absolute error (MAE), mean squared error (MSE), root mean squared error (RMSE), mean absolute percentage error (MAPE) and weighted average percentage error (WAPE). The chosen metrics were used as a basis for evaluating and comparing the effectiveness of the algorithms in each scenario. The respective formulas for these metrics are defined in Equations 8.1, 8.2, 8.3, 8.4 and 8.5, respectively. Mean Absolute Error (MAE). MAE reflects the average of the absolute differences between the actual and predicted observations in 124 Chapter 8. Models of predicting power supply and demand the test sample. It is calculated as follows: MAE =1 n n X j=1 |yi−ˆyj|,(8.1) where iand jare indexes, nis the number of observations, yjand ˆyj are the actual and the predicted values, respectively. Mean Squared Error (MSE). It measures the average of squared differences between actual and predicted observations. Using the same symbols as in MAE, the formulation is as follows: MSE =1 n n X j=1 (yi−ˆyj)2(8.2) Root Mean Squared Error (RMSE). It measures the square root of the average of squared differences between the actual and predicted observation and, using the previous symbols, RMSE can be formulated as follows: RMSE =v u u t 1 n n X j=1 (yi−ˆyj)2(8.3) Mean Absolute Percentage Error (MAPE). This metric evaluates the accuracy of the prediction of the forecasting model. It is formulated using the following equation: MAP E =1 n n X j=1 |yi−ˆyj| m´ax(ϵ, |yi|)(8.4) Weighted Average Percentage Error (WAPE). This metric evaluates the accuracy of the prediction of the forecasting model, taking into account the weights of the observations. It is formulated using the following equation: W AP E =1 n n X j=1 |yi−ˆyj| × wj m´ax(ϵ, |yi|), , (8.5) where wjrepresents the weight assigned to each observation. 8.2. Results This section discusses the results achieved through the use of the chosen forecasting models and the collected dataset. The objective was to evaluate 8.2. Results 125 and compare the performance of the different models for demand and supply datasets in various scenarios. Specifically, the objective is to select the best prediction model for each forecast horizon (24 h, 48 h, 72 h, and 168 h). For this purpose, the models were tested with seven different past history periods. Reproducibility of experiments can be performed with the code from the public repository located at [38]. Consistent with previous studies, five widely recognized metrics from the forecasting literature were selected to evaluate the models; all of them have been explained in the previous section. All implementations were carried out using the Python programming language. Before the experiments were carried out, it was necessary to test the trends and seasonality of the data. Figure 8.8 depicts the trend of demand and seasonality of the data. According to Figure 8.8, it could be observed that the data has trended at specific time series and seasonality features. The demand trend showed the general direction in which demand for power is moving over time. Clearly, some periods showed an increase or decrease in demand based on the period of time. However, seasonality in the figure refers to patterns in demand that repeat over time, such as higher demand for certain times during a season. The residuals represent the differences between the actual demand for power and the predicted demand. They represent the variability in demand that cannot be explained by trends or seasonality. The same behavior is observed in the supply data, as shown in Figure 8.9. Visualizing the demand and supply shows the fluctuation of power in Iraq. Figure 8.8: Demand trend and seasonality analysis. Figure 8.9: Supply trend and seasonality analysis. 126 Chapter 8. Models of predicting power supply and demand As announced in Section methodology is divided into two parts. The first step was designed to determine which hyperparameters are best for each model with a specific past history of 72 hours and a forecast horizon of 24 hours. Once these parameters and thus the best model have been identified, the model is trained on different scenarios in which the prediction horizon and past history vary. To make a more extensive comparison and to see the behavior of these data with different prediction models, it has been decided to use a total of six models, three of which are considered machine learning and the other three deep learning models. The models chosen were as follows: linear regression (LR), random forest (RF), XGBoost (XGB), long-short-term memory network (LSTM), temporal convolutional network (TCN), and multi-layer perceptron (MLP). The performance of these models is evaluated using the metrics MSE, RMSE, MAE, WAPE, and MAPE. The proposed methodology aims to establish an experimental framework for the prediction of energy supply and demand in Iraq. The following sections will present the results separately for the two-time series analyzed in this study, beginning with the demand and then the supply. 8.2.1. Energy demand forecasting Table 8.5 provides results for forecast horizon 24h and demand dataset. The results are divided into two main categories: more classical machine learning algorithms and deep learning models. Models were benchmarked and compared with respect to past history and five performance metrics. The two best-performing models are linear regression for machine learning models and TCN for deep learning. The first obtains MAE value of 450 and MAPE of 0.025% for linear regression, while the second gets MAE value of 493 and MAPE of 0.026%. Regarding the models with the worst results, it can be seen that the worst in the first category was obtained with random forest with MAE value of 870 and MAPE of 0.040%. In the second category, MLP was the worst, with a MAE value of 521 and MAPE of 0.028%. 8.2. Results 127 Table 8.5: Forecasting demand with a forecast horizon of 24h and different past history scenarios. The optimal outcomes of each model have been emphasized in bold. FORECAST 24 H DEMAND Model Past History MSE RMSE MAE WAPE MAPE MACHINE LEARNING LR 24 1083220 581.6717 502.9881 0.026775 0.027288 48 955023.4 544.9984 463.4439 0.024786 0.025514 72 902146.3 535.9016 454.1736 0.024341 0.025129 96 901062.8 534.4753 452.8799 0.024281 0.025125 120 893241.5 532.5168 450.8758 0.024165 0.025012 144 894859.1 535.2261 453.0205 0.02428 0.025122 168 897856.9 537.4003 455.4497 0.02441 0.025241 RF 24 1777895 1004.792 870.3464 0.040312 0.040828 48 1783794 1008.703 874.9128 0.040556 0.041089 72 1786500 1012.165 879.3987 0.040759 0.041282 96 1787945 1014.781 882.0955 0.040885 0.041402 120 1802335 1020.769 887.2238 0.041098 0.041599 144 1816157 1026.992 892.7668 0.041359 0.04186 168 1824805 1028.919 893.9624 0.041408 0.041909 XGB 24 1405847 797.0603 662.6271 0.030384 0.031097 48 1420858 781.875 655.0541 0.030094 0.030448 72 1381567 783.4709 661.1899 0.030483 0.030734 96 1465919 816.2758 690.202 0.031836 0.032145 120 1627843 872.1356 737.7352 0.033911 0.034171 144 1691088 898.1677 761.9106 0.035062 0.035242 168 1768986 928.5309 794.6021 0.036452 0.036505 DEEP LEARNING LSTM 24 1144054 638.9068 545.846 0.028696 0.029133 48 984984.4 627.0448 541.006 0.027617 0.028323 72 925205.1 595.9861 507.4458 0.026359 0.027138 96 910309.9 586.0333 501.2808 0.025987 0.026749 120 974763.3 628.0135 530.1516 0.027475 0.028304 144 953322.1 634.6448 545.7355 0.02779 0.028519 168 1030154 689.2143 591.5543 0.02997 0.030618 TCN 24 1152463 651.2944 560.162 0.029504 0.029947 48 1016793 607.09 512.1052 0.026991 0.027858 72 945475.2 583.6522 493.6367 0.026075 0.026877 96 1007753 611.8488 521.1543 0.027649 0.028704 120 1280490 811.7644 702.6569 0.035907 0.03724 144 1015919 649.027 553.7207 0.02878 0.029635 168 1122628 714.8256 604.2888 0.031649 0.032668 MLP 24 1210091 678.1158 589.9583 0.030914 0.031547 48 1027575 621.4995 521.9852 0.027398 0.028208 72 1016364 642.0646 548.8354 0.028629 0.029414 96 1089717 710.9572 596.4573 0.030895 0.03187 120 1011618 625.9169 529.1662 0.027787 0.028696 144 1046958 665.4122 562.4341 0.029188 0.030082 168 1003728 620.1257 522.4953 0.027718 0.028654 Table 8.6 provides results for forecast horizon 48h and demand dataset. The two best-performing models are linear regression for machine learning and LSTM for deep learning. The first obtains MAE value of 643 and MAPE of 0.035% while the second gets MAE value of 678 and MAPE of 0.036%. As for the models with the worst results, it can be seen that the worst in the first category was obtained by random forest with MAE value of 1002 and MAPE of 0.048%. In the second category, TCN was the worst, with MAE value of 685 and MAPE of 0.037%. 128 Chapter 8. Models of predicting power supply and demand Table 8.6: Forecasting demand with a forecast horizon of 48h and different past history scenarios. The optimal outcomes of each model have been emphasized in bold. FORECAST 48 H DEMAND Model Past History MSE RMSE MAE WAPE MAPE MACHINE LEARNING LR 24 1802788 810.2798 685.2839 0.036443 0.037186 48 1630950 773.3018 648.7147 0.034592 0.035509 72 1592371 766.11 643.1999 0.034326 0.035321 96 1594239 766.6057 643.9121 0.034368 0.035413 120 1593819 767.8873 644.3848 0.034365 0.035409 144 1594905 771.1242 647.1179 0.034486 0.035515 168 1594976 773.5811 649.7903 0.034629 0.035647 RF 24 2391115 1167.108 1002.596 0.047561 0.048195 48 2360015 1165.339 1003.585 0.047664 0.04828 72 2347167 1170.524 1009.928 0.047902 0.048457 96 2360748 1177.735 1016.408 0.048168 0.048714 120 2368809 1183.015 1020.23 0.048316 0.048844 144 2385238 1185.532 1022.206 0.048425 0.048967 168 2392329 1189.519 1025.292 0.048561 0.049098 XGB 24 2205185 1020.789 848.3964 0.039442 0.04015 48 2145587 994.6335 833.9088 0.038913 0.03917 72 2172596 1030.319 869.4493 0.040367 0.040629 96 2333627 1078.039 912.2416 0.042222 0.042534 120 2558740 1140.924 962.5131 0.04452 0.044707 144 2740464 1192.588 1011.527 0.046635 0.046754 168 2763566 1211.867 1035.271 0.047752 0.04777 DEEP LEARNING LSTM 24 1880940 871.5793 738.2946 0.038666 0.03924 48 1628318 805.7887 678.4556 0.035587 0.036495 72 1667014 862.0683 728.2714 0.037642 0.038521 96 1654046 848.2781 719.0411 0.038009 0.039209 120 1733785 902.0217 780.9808 0.039825 0.040674 144 1692101 883.8551 751.2145 0.038614 0.039472 168 1890780 996.885 838.3699 0.043165 0.044227 TCN 24 1889494 856.2178 723.4069 0.038303 0.039126 48 1710081 826.531 697.5408 0.036804 0.037784 72 1682362 817.2671 685.4659 0.036445 0.037561 96 1823857 934.5217 775.2336 0.040206 0.041349 120 1895499 949.2139 823.3499 0.04243 0.043457 144 1799922 878.4809 740.2786 0.039094 0.040355 168 2143434 1110.038 936.7238 0.047366 0.048361 MLP 24 1929685 891.4968 756.4653 0.039866 0.040713 48 1689021 816.0728 680.9173 0.036003 0.036978 72 1725579 858.2358 716.1583 0.037721 0.038835 96 1761828 841.7797 703.5016 0.037501 0.038748 120 1766339 870.6981 735.7779 0.038496 0.039554 144 1807041 909.212 772.0895 0.040092 0.041122 168 1819894 912.3422 760.1011 0.039903 0.041027 Table 8.7 provides the results for the forecast horizon 72h and demand dataset. The two best-performing models are linear regression for machine learning models and LSTM for deep learning. The first one obtains MAE value of 784 and MAPE of 0.042%, while the second gets MAE value of 815 and MAPE of 0.043%. As for the models with the worst results, it can be seen that the worst in the first category was obtained with random forest with MAE value of 1096 and MAPE of 0.053%. In the second category, MLP was the worst, with MAE value of 849 and MAPE of 0.045%. 8.2. Results 129 Table 8.7: Forecasting demand with a forecast horizon of 72h and different past history scenarios. The optimal outcomes of each model have been emphasized in bold. FORECAST 72 H DEMAND Model Past History MSE RMSE MAE WAPE MAPE MACHINE LEARNING LR 24 2343361 969.2559 815.0429 0.04338 0.044146 48 2194789 939.0349 787.3046 0.041973 0.042938 72 2163190 934.0767 784.3679 0.041838 0.042889 96 2179585 938.2074 787.8978 0.042021 0.043112 120 2188277 940.9846 789.8374 0.042087 0.043168 144 2183029 942.9451 791.3409 0.042136 0.043194 168 2179172 945.0924 793.0812 0.042228 0.043274 RF 24 2843364 1286.215 1096.141 0.052777 0.05327 48 2784749 1286.537 1099.299 0.052916 0.05339 72 2754846 1286.674 1100.026 0.052894 0.053328 96 2759947 1290.505 1102.892 0.053005 0.053428 120 2782343 1297.687 1109.155 0.053253 0.05366 144 2791838 1300.271 1110.829 0.053346 0.053752 168 2802719 1303.531 1113.451 0.053483 0.053874 XGB 24 2719869 1171.211 974.6917 0.045809 0.046326 48 2649420 1154.008 964.7845 0.045354 0.045535 72 2714473 1192.106 1001.543 0.046989 0.047144 96 2931581 1252.13 1052.446 0.049163 0.049332 120 3278267 1330.852 1123.34 0.052317 0.052419 144 3328298 1359.648 1154.938 0.053922 0.053924 168 3349189 1374.13 1168.922 0.054669 0.054564 DEEP LEARNING LSTM 24 2384366 1012.584 852.4807 0.044892 0.045506 48 2346997 1086.683 934.6706 0.047782 0.048505 72 2135899 968.358 815.0538 0.042992 0.043995 96 2191907 1000.249 839.9378 0.044161 0.045158 120 2452033 1114.831 961.8425 0.049228 0.049978 144 2525372 1128.269 941.3932 0.049361 0.050675 168 2314688 1050.92 890.9157 0.046523 0.04742 TCN 24 2412693 1007.948 843.7587 0.044764 0.045463 48 2359442 1087.277 939.75 0.048301 0.049127 72 2326596 1065.875 916.8954 0.047433 0.048398 96 2354439 1050.395 891.3047 0.046665 0.047749 120 2388654 1072.106 920.5687 0.047755 0.048674 144 2405361 1082.162 908.5797 0.047351 0.048368 168 2385619 1055.99 901.3585 0.047314 0.048281 MLP 24 2406789 1018.402 852.6924 0.045053 0.045732 48 2530987 1082.555 924.6539 0.048367 0.049807 72 2277146 1029.653 873.125 0.045628 0.046588 96 2340006 1015.634 849.3662 0.044998 0.046293 120 2382516 1052.136 892.1342 0.04671 0.047733 144 2510326 1123.572 956.0905 0.049549 0.050525 168 2306154 1020.625 858.7795 0.045353 0.046381 Table 8.8 provides the results for the 168h forecast horizon and the demand data set. The two best-performing models are linear regression for machine learning models and MLP for deep learning. The first obtains the MAE value of 1123 and MAPE of 0.060%, while the second gets MAE value of 1162 and MAPE of 0.062%. As for the models with the worst results, it can be seen that the worst in the first category was obtained with random forest with MAE value of 1368 and MAPE of 0.068%. In the second category, LSTM was the worst, with MAE value of 1187 and MAPE of 0.063%. 130 Chapter 8. Models of predicting power supply and demand Table 8.8: Forecasting demand with a forecast horizon of 168h and different past history scenarios. The optimal outcomes of each model have been emphasized in bold. FORECAST 168 H DEMAND Model Past History MSE RMSE MAE WAPE MAPE MACHINE LEARNING LR 24 3800349 1383.785 1136.715 0.060547 0.0614108 48 3729226 1368.163 1123.062 0.059875 0.060936 72 3751450 1372.508 1128.569 0.060209 0.061336 96 3801594 1382.574 1137.932 0.060724 0.061882 120 3806957 1385.469 1139.332 0.060763 0.061900 144 3786811 1384.241 1136.500 0.060546 0.061643 168 3779494 1384.868 1136.100 0.060512 0.061589 RF 24 4132224 1645.646 1368.324 0.067707 0.068097 48 4132224 1645.646 1368.324 0.067707 0.068097 72 4124540 1658.615 1379.626 0.068147 0.068412 96 4148383 1666.749 1387.417 0.068480 0.068708 120 4185005 1676.267 1395.212 0.068866 0.069091 144 4228470 1685.671 1401.884 0.069175 0.069382 168 4228470 1685.671 1401.884 0.069175 0.069382 XGB 24 4238792 1608.761 1316.703 0.063256 0.063539 48 4455385 1664.261 1363.103 0.065114 0.065111 72 4673260 1724.463 1422.586 0.067710 0.067614 96 4894699 1770.473267 1466.999 0.069817 0.069655 120 5111436 1819.981 1507.251 0.072056 0.071793 144 5177563 1846.604 1530.110 0.073505 0.073266 168 5288407 1872.017 1549.172 0.074731 0.074559 DEEP LEARNING LSTM 24 3844556 1444.743 1192.793 0.062690 0.063439 48 3793894 1443.222 1187.218 0.062465 0.063381 72 3744980 1473.913 1231.977 0.063936 0.064635 96 3834768 1467.208 1215.460 0.063787 0.064650 120 3866136 1437.989 1192.848 0.063318 0.064419 144 4082703 1586.656 1313.151 0.068273 0.069206 168 3809797 1476.846 1216.722 0.064437 0.065349 TCN 24 3832966 1462.073 1203.622 0.063298 0.063888 48 4227610 1670.172 1430.471 0.072372 0.072844 72 3868559 1421.634 1175.009 0.062504 0.063839 96 4098126 1470.115 1217.771 0.064670 0.066255 120 3998622 1456.357 1209.385 0.064342 0.065665 144 4040086 1494.787 1231.038 0.065116 0.066609 168 4233132 1496.094 1254.320 0.066543 0.068311 MLP 24 3984785 1424.518 1184.110 0.062992 0.064197 48 3785355 1412.873 1162.771 0.061658 0.062676 72 3917988 1408.398 1169.994 0.062392 0.063881 96 3961381 1440.025 1189.306 0.063348 0.064654924 120 3982736 1438.945 1195.718 0.063720 0.065118 144 4174662 1573.977 1290.176 0.067576 0.068745 168 3938162 1445.909 1197.005 0.063666 0.064913 Figure 8.10 provides an overview of the previous tables, taking into account the MAE metric. This figure groups the results according to the forecast horizon, showing a comparison between each of the models used in the experimentation. It is interesting to note at first glance how, as the prediction horizons increase, the results get worse for all models. This decrease may be mainly due to the fact that higher prediction horizons imply greater difficulty in prediction. On the basis of this figure, it seems that the linear regression model outperformed the other models, with the lowest MAE across all four forecast horizons. However, random forest seems to almost always obtain the worst results, although if not only the individual results are taken into account, but also the median of the distribution, the worst model 8.2. Results 131 would be RF. Finally, it is worth noting that there is a certain difference in results between the DL and ML models used, the latter being the ones that obtain greater variability between models, which may be due to the fact that DL models in this case are more suitable for capturing the temporal characteristics of the data set. Figure 8.10: Distribution of results in terms of MAE grouped by forecast horizon. Figure 8.11 shows the comparison between the predicted and actual values for the last month of the study. The best model among all the predictions of the experiments served as the basis for the comparison. In this case, the best result was obtained with a prediction horizon of 24 hours with the linear regression model. Figure 8.11: Comparison between actual and predicted values for demand in the last month of the data set. The observed results are those obtained with the best model. The Friedman test has been applied to assess the overall significance of differences among the performances of the six models. The Friedman test is a non-parametric test used to compare three or more matched groups (in this case, models) without assuming that the data follow a specific distribution, and the objective is to determine whether there are significant differences between the groups. In this context, each model provides 28 evaluation values (7 different past values for 4 different forecast windows) from tables