Stochastic Model Predictive Control and Machine Learning for the Participation of Virtual Power Plants in Simultaneous Energy Markets

Proyecto Fin de Carrera Ingeniería de Telecomunicación Formato de Publicación de la Escuela Técnica Superior de Ingeniería Autor: F. Javier Payán Somet Tutor: Juan José Murillo Fuentes Dep. Teoría de la Señal y Comunicaciones Escuela Técnica Superior de Ingeniería Universidad de Sevilla Sevilla, 2013 Tesis Doctoral Ingeniería automática, electrónica y de telecomunicación Stochastic Model Predictive Control and Machine Learning for the Participation of Virtual Power Plants in Simultaneous Energy Markets Autor: Juan Aguilar Guisado Directores: Carlos Bordons Alba y Alicia Arce Rubio Ingeniería de Sistemas y Automática Escuela Técnica Superior de Ingeniería Universidad de Sevilla Sevilla, 2022 Tesis Doctoral Ingeniería automática, electrónica y de telecomunicación Stochastic Model Predictive Control and Machine Learning for the Participation of Virtual Power Plants in Simultaneous Energy Markets Autor: Juan Aguilar Guisado Directores: Carlos Bordons Alba Alicia Arce Rubio Ingeniería de Sistemas y Automática Escuela Técnica Superior de Ingeniería Universidad de Sevilla 2022 A Cristina, por el tiempo robado y por hacerme feliz. A Juan y Esperanza, por moldear lo que soy y ofrecérmelo todo sin condiciones. A Javier, por su amistad eterna y su amor de hermano. Acknowledgment Siento la más profunda necesidad de agradecer la aportación de aquellos que me han acompañado en este trayecto de manera directa o indirecta pero que, en definitiva, han colaborado en la elaboración de este trabajo. En primer lugar, me gustaría agradecer toda la dedicación, paciencia y orientación a mis directores de tesis: Carlos y Alicia. Carlos ha sido la orientación, el apoyo y la corrección constante, hasta el menor de los detalles. Es inimaginable tal dedicación y compromiso no solo a nivel técnico, sino también desde el prisma del desarrollo profesional y personal. Conocerlo y trabajar con él han sido de las experiencias más enriquecedoras de mi carrera académica y profesional. Muchas gracias, Carlos. Alicia es mi madre científica. Su excelencia técnica, su ambición y su perseverancia han sido un pilar. No puedo más que agradecer su confianza, valorar el tiempo que me ha regalado y todo lo que he conseguido aprender. Juntos contamos los proyectos por éxitos, sin duda, gracias a ella. Me gustaría destacar mi más sincero agradecimiento a Ricardo Galán por impulsarme, animarme y convencerme a empezar este trabajo, a pesar de las consecuencias que suponía. Su experiencia, sus consejos y su confianza desde mis primeras horas como profesional han contribuido al ingeniero que soy. No hubiera sido imaginable sin él. A Cristina, y a mis padres y hermano. Han sido el soporte sobre el que me he ido apoyando a lo largo de estos años en los que ha habido momentos para todo. No me dejaron abandonar. Particularmente, gracias a ti, mamá, por el tiempo dedicado y las concienzudas correcciones que han permitido la presentación de la tesis en inglés, planteando las preguntas adecuadas para asegurar que, lo que parecía un problema de idioma, no era una falta de precisión en la exposición argumental. Lo hemos trabajado mucho tiempo. Por último, realizar una especial mención a Andrés, Pedro y Rubén, porque es un lujo poder contar con su amistad y compañía profesional durante tantísimo tiempo, a pesar de la distancia que ha habido durante intervalos de tiempo que siempre terminan acabando. Juan Aguilar Guisado Sevilla, 2022 III Resumen La irrupción de los sistemas de generación distribuidos en los sistemas eléctricos dan lugar a nuevos escenarios donde los consumidores domésticos (usuarios finales) pueden participar en los mercados de energía actuando como prosumidores. Cada prosumidor es considerado como un nodo de energía con su propia fuente de generación de energía renovable, sus cargas controlables y no controlables e incluso sus propias tarifas. Los nodos pueden formar agregaciones que serán gestionadas por un agente denominado operador del sistema. La participación en los mercados energéticos no es trivial, bien sea por requerimientos técnicos de instalación o debido a la necesidad de cubrir un volumen mínimo de energía por transacción, que cada nodo debe cumplir individualmente. Estas limitaciones hacen casi imposible la participación individual, pero pueden ser salvadas mediante participaciones agregadas. El agregador llevará a cabo la ardua tarea de coordinar y estabilizar las operaciones de los nodos de energía, tanto individualmente como a nivel de sistema, para que todo el conjunto se comporte como una unidad con respecto al mercado. Las entidades que gestionan el sistema pueden ser meras comercializadoras, o distribuidoras y comercializadoras simultáneamente. Por este motivo, el modelo de optimización sobre el que basarán sus decisiones deberá considerar, además de las tarifas agregadas, otras individuales para permitir facturaciones independientes. Los nodos deberán tener autonomía legal y técnica, así como el equipamiento necesario y suficiente para poder gestionar, o delegar en el operador del sistema, su participación en los mercados de energía. Esta agregación atendiendo a reglas de negocio y no solamente a restricciones de localización física es lo que se conoce como Virtual Power Plant. La optimización de la participación agregada en los mercados, desde el punto de vista técnico y económico, requiere de la introducción del concepto de virtualización dinámica del almacenamiento, para lo que será indispensable que los nodos pertenecientes al sistema bajo estudio consten de una batería para almacenar la energía sobrante. Esta virtualización dinámica definirá particiones lógicas en el sistema de almacenamiento para dedicar diferentes porcentajes de la energía almacenada para propósitos distintos. Como ejemplo, se podría hacer una virtualización en dos particiones lógicas diferentes: una V XII Glossary Pgen,k(t)Forecasted generation power for node kas time t(kW). Pload,k(t)Forecasted load power for node kas time t(kW). SOC State of Charge. TsSample time. ηcharge,kCharge efficiency for node k. ηdischarge,kDischarge efficiency for node k. ψgenkGeneration Neural Network agent in the Disturbance Mitigation Layer for node k. ψloadkLoad Neural Network agent in the Disturbance Mitigation Layer for node k. Pricein j,k(t)Consumption price (€/kWh) of node kat time t. Pricein j,k(t)Injection price (€/kWh) of node kat time t. ϕ−1 genk,t(δ) Result of applying the Chance-Constrained models for generation for the node kat t, with a risk factor of (1−δ). iIntrahour sample time execution index, from 0 to 5. rkReliability Factor for node k. tSample time index. tDRnSample time of the nth execution of the DR algorithm. Acronyms AGG-DA Aggregated Day-Ahead. AI Artificial Intelligence. API Application Programming Interface. AS Ancillary Service. BESS Battery Energy Storage System. BRNN Bidirectional Recurrent Neural Network. CAES Compressed Air Energy Storage. CC Chance Contraints. CC-MPC Chance Contraint Model Predictive Control. CDF Cumulative Distribution Function. CNN Convolutional Neural Network. DA Day-Ahead. DAM Day-Ahead Market. DER Distributed Energy Resource. DES Distributed Energy Storage. DG Distributed Generation. DIP Dynamic Intent Profile. DL Deep Learning. DML Disturbance Mitigation Layer. DR Demand Response. XIII XIV Acronyms DRF Django Rest Framework. DRP Demand Response Program. DSO Distribution System Operator. DSV Dynamic Storage Virtualization. ECDF Empirical Cumulative Distribution Function. ED Encoder-Decoder Model. EMP Energy Management Platform. EMS Energy Management System. EN Energy Node. ESO Electricity System Operator. ESS Energy Storage System. EU European Union. FDR Fast Demand Response. FES Flywheel Energy Storage. FF Feed Forward. GAN Generative Adversarial Networks. Genco Large generation plant. GRU Gated Recurrent Unit. IoT Internet of Things. IP Intent Profile. ISO Independent System Operator. JSON JavaScript Object Notation. KPI Key Performance Indicator. LSTM Long-Short Term Memory. MILP Mixed Integer Linear Programming. MIQP Mixed Integer Quadratic Programming. ML Machine Learning. MLD Mixed Logic Dynamic. Acronyms XV MO Market Operator. MPC Model Predictive Control. MS Market Settlement. MSE Mean Squared Error. NLP Natural Language Processing. NN Neural Network. ORM Object-Relational Mapping. P2P Peer to Peer. PA Profile Accuracy. PDDR Price Driven Demand Response. PDF Probability Distribution Function. PHES Pumped Hydro Energy Storage. PR Penalty Reduction. PV Photovoltaic. ReLU Rectified Linear Unit. RES Renewable Energy Resource. REST REpresentational State Transfer. RNN Recurrent Neural Network. RTB Real Time Balancing. SaaS Software as a Service. SG Smart Grid. SMES Superconducting Magnetic Energy Storage. SMPC Stochastic Model Predictive Control. SO System Operator. SQL Structure Query Language. TES Thermal Energy Storage. TSO Transmission System Operator. VPP Virtual Power Plant. Contents 1 Introduction 1 1.1 Virtual Power Plants, Smart Grids and Distributed Energy Resources Management 3 1.2 Electricity markets: Retail and Wholesale 5 1.2.1 Wholesale Competition 6 1.2.2 Retail Competition 6 1.3 Current Challenges in the operation of Virtual Power Plants 7 1.4 Thesis Motivation 8 1.5 Research Objectives 8 1.6 Dissertation Layout 9 1.7 Publications 10 1.7.1 Analysis of Data Generated by an Automated Platform for Aggregation of Distributed Energy Resources 10 1.7.2 Chance Constraints and Machine Learning integration for uncertainty management in Virtual Power Plants operating in simultaneous energy markets 10 1.7.3 Intent Profile Strategy for Virtual Power Plant Participation in Simultaneous Energy Markets With Dynamic Storage Management 10 2 Literature Review 13 2.1 Introduction 13 2.2 Distributed Energy Resources 14 2.2.1 Technologies 15 2.2.2 Applications 16 Standalone 16 Peak Shaving 17 Standby Power 17 XVII XVIII Contents Load Curtailment 17 Demand Response 17 2.2.3 Storage Methods 17 2.3 Energy Market Participation 18 2.3.1 Day Ahead Market Participation 18 2.3.2 Demand Response Programs 20 2.3.3 Operation Strategies 22 2.4 Intelligent Agents 23 2.4.1 Problems classification 24 Fully vs Partially Observable 24 Deterministic vs Stochastic 25 Discrete vs Continuous 25 Benign vs Adversarial 25 2.4.2 Neural Networks and Machine Learning 25 Universality in Neural Networks 26 2.4.3 Neural Networks Classification 26 Simple Perceptron and Multilayer Perceptron 26 Convolutional Neural Networks 27 Autoencoders 29 Recurrent Neural Networks 29 Long Short Term Memory Networks 30 Gated Recurrent Unit Networks 31 Generative Adversarial Networks 32 Neural Networks in the scope of this dissertation 33 3 System model and Optimal Operation Strategy definition 35 3.1 System Definition 36 3.1.1 Energy Node 36 3.1.2 Storage 37 3.1.3 Status 37 3.1.4 Reliability Factor 38 3.2 Forecasts 38 3.3 Baseline and Piped Baseline 38 3.4 Profile Accuracy 40 3.5 Intent Profiles 40 3.6 Dynamic Storage Virtualization 40 3.6.1 Individual or aggregated Intent Profiles 42 3.7 Framework Definition: A general overview 42 4 Optimal Virtual Power Plant multi-service participation 45 4.1 Bidding Optimization for Day-Ahead Market Participation 45 4.1.1 Mixed Logic Dynamic Systems 46 4.1.2 Optimization Scenarios 47 Contents XIX 4.1.3 Algorithm inputs 47 4.1.4 Simulations and submissions 48 4.1.5 Objective and Cost function 48 4.1.6 Linear State Space Model 49 4.1.7 Model Specification 49 4.1.8 System Constraints 50 Power Battery physical limits 50 State of charge limits 50 Aggregation limits 51 4.1.9 Cases of Study 51 Evaluation of the different Day Ahead model versions 51 Intent profiles evaluation 52 Analysis of the results given different Intent Profiles 53 Evolution of the State of Charge values given different Intent Profiles 54 4.2 Penalty Reduction Optimization 55 4.2.1 Objective and cost functions 57 4.2.2 System Constraints 57 Grid constraints 57 Energy node constraints 58 4.2.3 Fault Detection Enhancement 58 Intrahour equilibrium strategy 59 Non controllable nodes 60 Meeting injection 62 The importance of the injection meeting enhancement during operation 63 4.3 Demand Response Program Integration 63 4.3.1 Objective and Cost function 64 4.3.2 Specific System Constraints 65 Profile limits 65 Feeding compensation vs No-Feeding compensation 65 4.3.3 Fast Demand Response (Maneuvers) 65 4.4 Conclusions 66 5 Chance Constraint Optimization 69 5.1 Disturbance Mitigation Layer 70 5.1.1 Disturbance Mitigation Layer formulation in the framework 71 Stochastic Component Service 72 5.1.2 Chance Constrained Disturbance Mitigation Service 72 Cumulative Distribution Function Definition and implementation 74 5.1.3 Neural Network Disturbance Mitigation Service 75 Neural Network Model Definition 75 XX Contents 5.2 Results 76 5.2.1 Disturbance Predictors 77 5.2.2 Case Study: Disturbance Mitigation Layer (DML) Performance Assessment 77 5.2.3 Disturbance Mitigation Layer Robustness 80 5.3 Conclusions 81 6 The Pilot. Analysis in a real environment 83 6.1 Pilot Description 83 6.2 Generic Operation Information 84 6.3 Performance Testing 87 6.4 Robustness Assessment 89 7 Framework as a platform 91 7.1 Design Patterns and Software Architecture 92 7.2 Platform Definition 93 7.2.1 Layered Architecture Component 94 7.2.2 Service-Based Architecture Component 95 7.3 Software Model Design 96 7.4 Asynchronous Execution 99 7.5 Remote Solvers 99 8 Conclusions 101 8.1 Closing Remarks 101 8.2 Contributions 101 8.3 Future Studies 102 List of Figures 105 List of Tables 109 Listings 111 Bibliography 113 1 Introduction With great power comes great responsibility Ben Parker The world, as we know it, is totally dependent on electricity and power systems. However, the industry has not been in the need of defining a total revolution regarding the way the energy is generated, transported and consumed. This is due to the good relation between the economic and reliability factors presented by classical electrical systems. The big picture of electrical systems has somehow evolved more slowly than other disciplines in which external agents, such as the leading digital revolution, have forced fast and vertiginous changes. In addition, making any revolution in this field is usually extremely complicated, since power systems are involved in almost any critical system, as well as some inherited technical impediments, such as the need to work in the continuous spectrum. Figure 1.1 shows a simplified version of a classic electrical system. The primary energy (energy which is directly harvested from natural resources) is generated in central power plants and it is distributed to end-users through some energy transmission systems called power lines. This energy is converted before and during the transmission stages so that it could be usable to end-users. Some examples of energy conversion technologies could be the hydrocarbon molecules in the coal (chemical stored energy → heat), turbines (heat → motion → electricity) or even the electric circuit with a battery (chemical energy → electricity). The transmission system contains a set of substations which act as switching points with different functions. The substations can be classified as: • Step-up Transmission Substation: It uses large power transformers to distribute the electrical power to distant locations by increasing the voltage. • Step-down Transmission Substation: These substations change the transmission voltage to subtransmission voltage and prepare the eletric power to be distributed. 1 8Chapter 1. Introduction •Multi-Objective. More details on the current state of the art of these challenges can be found in the chapter Literature Review. The only ones which are out of the scope of this thesis are those classified in reactive power and stability. The others have been covered theorically or empirically in this dissertation. 1.4 Thesis Motivation The optimal management of VPPs must handle a large number of variables with complex models. Experts are able to intuit the selection of one of the best solutions most of the times, but this might not be enough for some business models, or not even enough for problems in which the decision must be taken before gathering all the necessary information, and the solution is based on forecasts. The problem becomes more difficult when the VPP is involved in different services and these must be coordinated and synchronized. The solution must not only be efficient, but also robust and technically possible. This will involve various technical skills from different disciplines so that the research could result in a product which is close to the industry. 1. Expert SOs need help in handling the difficult problem of optimizing the operation of the VPP. This help is necessary to optimize the bids before participating in markets, as well as to control the grid at the operation stage. 2. It is economically beneficial to have multi-purpose VPPs to run better business models. It is possible to achieve the multi-purpose without impacting other services. 3. The combination of several disciplines makes it possible to build incredible intelligent agents. In addition, it is necessary to demonstrate that the theory can be applied in the field of research. Research leads to innovation, and both are the key to create a better future. In this thesis, several disciplines have been combined, such as electrical, control and software engineering, to build and deploy a real platform which meets the time constraints, the computational complexity and the necessary standards to be served as a service. 1.5 Research Objectives The main objective of this thesis is to develop an intelligent agent which helps and improves the VPP operator’s performance when participating in multiple markets simultaneously. The proposed VPP considers different ENs with photovoltaic generation, non-regulable loads and an energy storage system composed of a battery which acts as a buffer. The dissertation has the following specific objectives: 1. To improve the operation of VPPs in energy markets to obtain the benefits of not wasting clean energy by operating optimally with the excess energy in these energy markets. 1.6 Dissertation Layout 9 2. To present and develop the concept of DSV and dynamic Intent Profiles (IPs) for VPPs to offer multiple simultaneous energy services at the same time. 3. To develop an optimal economic management model by using Chance Contraint Model Predictive Control (CC-MPC), to ensure optimal and flexible solutions for the operation of the VPP where there are some problems related to a possible lack of accuracy in the forecastings. 4. To implement a scalable and robust platform to be deployed and tested in real environments. 1.6 Dissertation Layout The optimal operation of VPPs implies different challenges which were not present in classical power systems. Decentralization means distributed environments so better communications and more robust solutions in terms of failures are needed. This dissertation defines an approach to solve the difficult task of setting the optimal operation of a VPP and of providing multiple services by participating in several markets simultaneously by controlling individual nodes, called prosumers. The dissertation is structured in eight chapters. In the first chapter , an introduction to the electricity system and electricity markets is provided to set the context of the research. In the second chapter there is a literature review which focuses on four main aspects: DERs state of the art, important aspects related to Day-Ahead Market (DAM) participations, several operation strategies to operate VPPs in energy markets, as well as some literature about intelligent agents. The system model and operating strategies are defined in the third chapter . In addition, the key concepts which are necessary to define the strategies have also been defined. The fourth chapter presents the implementation of strategies for multiple services and simultaneous participation in different energy markets. A reconfigurable multilevel dynamic control strategy for operating the grid by a Mixed Logic Dynamic (MLD) is proposed. The use of IPs, together with the DSV, improves the profitability of the operation as well as it reduces the potential penalties at operation time. The Model Predictive Control (MPC) for DA and DRP optimizations is solved as Mixed Integer Linear Programming (MILP). The PR algorithm to control the operation is solved as Mixed Integer Quadratic Programming (MIQP). In order to reduce the impact of deviations in forecasts, the inclusion of CC is detailed in the fifth chapter . Deterministic approaches have a strong dependency on forecast systems and the inclusion of chance constraints adds an extra flexibility, which improves most of the scenarios that the VPP may present. The CC factor is calculated as a combination of statistical methods and Artificial Intelligence (AI). The AI agent has been developed with an encoder-decoder architecture. The sixth chapter shows the results of a real pilot. The application was run and deployed in a real environment, integrated with physical devices. This experience demonstrated that it is possible to deploy a platform as the one proposed in the other chapters. The seventh chapter defines the details of the software: the architecture, the software model and the implementation details regarding deploying the algorithms as a service. 10 Chapter 1. Introduction The last chapter summarizes the main conclusions. In addition, the main contributions and future research lines are also explained. 1.7 Publications This dissertation yielded several publications from different parts of the research process, the software implementation and the pilot. Part of the work was developed within the framework of Netfficient (H2020 project), so it was necessary to filter all the outputs that involved third-party companies or research centers. This limitation reduced the number of possible publications to three valuable papers: one for an international congress and two for different international journals with a high impact factor. 1.7.1 Analysis of Data Generated by an Automated Platform for Aggregation of Distributed Energy Resources The paper [ 17 ] published the analysis of the data generated during the pilot operation deployed in the island of Borkum. The analysis of real data raised valuable conclusions about the feasibility and robustness of the strategy implemented for participating in simultaneous energy markets. •Congress: International Conference on Optimization and Learning •Book: Optimization and Learning •Publisher: Springer International Publishing •Date: February, 2020 1.7.2 Chance Constraints and Machine Learning integration for uncertainty management in Virtual Power Plants operating in simultaneous energy markets The paper [ 18 ] presented the DML implementation and the promising results of the combined Machine Learning (ML) and CC-MPC strategy to trade off the optimal DA participations and the potential penalties at the operation stage. The feed-forward disturbance compensation layer for the CC strategy with the encoder-decoder architecture was presented and assessed. •Journal: International Journal of Electrical Power & Energy Systems •Volume: 133 •Publisher: Elsevier •Date: December, 2021 1.7.3 Intent Profile Strategy for Virtual Power Plant Participation in Simultaneous Energy Markets With Dynamic Storage Management The paper [ 19 ] published the approach of using strategies based on DIPs combined with DSV to optimize the bids for simultaneous energy markets notonly according to economical 1.7 Publications 11 indicators, but also according to some risk factors. The publication includes case studies with satisfactory conclusions in non-ideal scenarios where it can be concluded that the combination of DIPs and DSV optimizes the participation at both bid and operation times and they improve the flexibility of the powerful two-stage hierarchical formulation methodology present in the current state of the art. •Journal: IEEE Open Access •Volume: 10 •Publisher: IEEE •Date: February, 2022 2 Literature Review It really matters whether people are working on generating clean energy or improving transportation or making the Internet work better and all those things. And small groups of people can have a really huge impact. Larry Page 2.1 Introduction Clean and distributed energy systems are becoming more and more important. Climate change and its consequences have become one of the major concerns worldwide. The developed countries must lead the society towards the transition from non-renewable energy generation sources to low-carbon ones, sharing a weighted responsibility with the developing countries playing a secondary role. The scenario in these countries is more problematic, since they have to balance the lack of having a regular access to electrical power with this shared responsibility [20]. According to [ 21 ], energy sources replace each other in a regular fashion. Oil replaced coal as well as coal replaced biomass. The next turn is for clean energy to success. Renewable energy accounted for 30.7 % of gross final electricity consumption, 19.5 % of energy consumption for heating and cooling, and 7.6 % of transport fuel consumption in the whole EU [ 22 ] by the end of 2017. The analysis of these data yields different possible interpretations since these volumes might seem to be not enough, but the trend is that these percentages are dramatically increasing. The share of renewable energy in gross final energy use in the EU has doubled since 2005. It reached 17.6 % in 2017, 18.0 % in 2018 and it increased further to the 22.1% in 2020 according different reports of the European Environment Agency. Thus, the EU reached its headline target (20%) for 2020. The current energy target has been set to 32% for 2030 [ 23 ]. Figure 2.1 shows EU and Member State shares of renewable energy sources and two trajectories with the objectives which were set for the year 2020 by the Renewable Energy Directory and the Renewable 13 14 Chapter 2. Literature Review Figure 2.1 Progress towards renewable energy source targets at Member State and EU-28 levels (source [23]). Energy Action Plan. Although this figure shows the aggregated data of all the countries, the report also presents disaggregated information per country. However, the success of DERs is not only related to their clean nature. Distributed systems are positioned as the perfect alternative also in the energy generation field since the centralized ones always require higher investments in infrastructures as well as in control systems, and they also present higher maintenance costs as well. The main objective of the development and deployment of DERs for the end-users is self-consumption. However, participating in energy markets can also be profitable if several prosumers participate in an aggregated way with the appropriate feed-in tariff [ 24 ]. In section 2.3, the two energy markets participations which are in the scope of this dissertation have been defined. Despite the fact that only DA and DRP participations have been considered, the proposed solution is open to the inclusion of other services for simultaneous operations. In section 2.3.3 different strategies for the optimal management of microgrids and VPPs are presented. Finally, the last objective is to get intelligent agents involved in the process of working with classical deterministic methods to get the best from both approaches. Section 2.4 describes which are the state-to-art techniques in artificial intelligence. 2.2 Distributed Energy Resources VPPs have traditionally had many different means of producing energy with a clearly decreasing trend of the use of non-renewable ones. Their environmental negative effects and the finite fossil fuel reserves are among the main driving factors which will inevitably put an end to the use of non-renewable generation sources. In fact, the World Energy Council has predicted that the global power output will increase from 23% as it was in 2010 to 34% in 2030 [ 25 ]. Moreover, according to the International Energy Agency, by 2.2 Distributed Energy Resources 15 Figure 2.2 DER technologies (source [27]). 2040, the total renewable electricity generation will be equal to that of the coal and natural gas-based electricity generation [26]. 2.2.1 Technologies DERs can be installed at consumers’ houses or at electric utility facilities, which reduces physical and electrical distances between the generation and the load. This reduces the losses, minimizes carbon emissions, and reschedules the establishment of new large generators and transmission lines [ 27 ]. In fact, the losses have a huge impact on the optimal economic dispatch and they determine the network configuration since they can behave as bottlenecks [28]. Another important advantage to consider is the fact that it is possible to provide remote locations with power by using DERs, where it is too expensive or impossible to build transportation lines from primary generators working as standalone power systems [ 29 , 30 ]. DERs are composed of Distributed Generations (DGs) units as well as Distributed Energy Storages (DESs). Figure 2.2 shows the different technologies of these components. Renewable DGs units in general, and photovoltaic generation with chemical storage units in particular, are the DGs technologies in the scope of the study. The use of Photovoltaic (PV) generation units has been increasing due to the continuous an dramatically decrease of the investment costs. In fact, the cost of solar panels has dropped by nearly 50% since 2014 [ 31 ]. In addition, it is a well known technology which has been implemented for almost one century, after the first discovery of the photovoltaic effect present in semiconductor materials two centuries ago. Table 2.1 summarizes a brief hitsorical timeline of the development of photovoltaic technology for electricity generation. One of the Key Performance Indicators (KPIs) to assess DGs technologies is efficiency. Efficiency can be defined as the amount of potential energy that becomes electricity. Most of the renewable generation sources present a low efficiency, as it can be seen in table 2.2. Despite its low maximum theoretical efficiency limit (33% according to [ 33 ]), PV generation is the most widely spread technology installed at prosumers’ due to its 16 Chapter 2. Literature Review Table 2.1 Brief historical timelines of the development of photovoltaic technology (source: [32]). Year Event 1839 Discovery of photovoltaic effect by Alexandre-Edmund Becquerel 1876 Demonstration of photovoltaic effect on selenium bar by William Grylls Adams and Richard Evans Day 1883 Construction of the first solar cell by using the semiconductor selenium with gold by Charles Fritts 1941 Patent for p-n junction solar cell using semiconductor silicon was applied by Russel Shoemaker Ohl for Bell Telephone Laboratories 1954 First practical silicon solar cell with an efficiency of 6% was constructed by Daryl M. Chapin, Callvin S. Fuller and Gerald L. Pearson for Bell Laboratories. Table 2.2 Comparison between the efficiencies of renewable generation sources (source: [34]). Solar Minimum Efficiency Maximum Efficiency Solar 14% 33% Wind 24% 55% Geothermal 10% 20% Hydro - 90% Biomass 20% 25% affordable investment costs, the size of the installs and its independence in terms of location requirements. 2.2.2 Applications DERs can be used for different purposes depending on commercial, technical or environmental considerations. Standalone A standalone power system does not need any connection to the utility grid. In addition, it can work autonomously. Sometimes, these systems are very important in remote areas with no available connection or when the cost of transporting the power is not cost effective. In addition, an ordinary power system can also work in this mode due to economical reasons and disconnect on demand to perform its business models. Standalone systems usually combine different DER technologies to generate reliable power. Operating and designing a standalone power system is a hard problem to solve, from small power systems such as 3 kWh/day housing units [35] to medium-large power systems [36, 37, 38]. 2.2 Distributed Energy Resources 17 Peak Shaving The randomness and volatility of renewable energy has a big impact in the stability of power systems. DERs, and their DESs can aliviate the peak load regulation pressure, and effectively achieve a load translation in time and space [ 39 ]. Since the stability of a power system depends on the equilibrium between supply and demand, DERs can solve the two main concerns: when the power supply is higher than the demand and vice versa, by increasing or decreasing their power setpoints commanded to the utility grid. Standby Power Standby power is ready-to-use electrical power without any current allocation in markets or stored for some consumption. Implementing DERs in a VPP allows the availability of power when the power supply is interrupted temporarily. However, the standby power can also be used to trade with markets or to participate in energy markets offering some services such as peak shaving or Demand Response (DR). The main purpose of storing energy will depend on economical or technical factors, which will be defined by the SO according to a established business model. Load Curtailment Load curtailment defines operations where there is a need of an unusual reduction in the consumption of a power system due to problems with the system stability or market requirements. It is somehow the opposite to peak shaving and both are related to DR actions. This reduction can be voluntary, based on incentives and penalties; or mandatory, to avoid a possible massive blackout. Demand Response DR defines the process of decreasing the amount of power required by customers from the utilities to strategically reduce the cost of energy, to ensure the stability of the system or for other market requirements. 2.2.3 Storage Methods RESs present a big issue related to their reliability and steadiness due to the large and sudden load variations, so the storage of energy becomes necessary. Thus, it is necessary to convert the energy to other forms, such as chemical or mechanical energy among many others to store electrical energy. A broad categorization of different methods of Energy Storage Systems (ESSs) is: • Mechanical: Pumped Hydro Energy Storage (PHES), Compressed Air Energy Storage (CAES) or Flywheel Energy Storage (FES). •Chemical: Fuel cell and electrolyzer. •Electro-chemical: Battery Energy Storage System (BESS). •Electro-static: Ultracapacitor, supercapacitor. •Electro-magnetic: Superconducting Magnetic Energy Storage (SMES). • Thermal: Sensible Thermal Energy Storage (TES), latent TES, thermochemical TES or pumped TES. 24 Chapter 2. Literature Review Figure 2.8 Different uncertainty sources. Figure 2.9 Perception-Action Cycle. Generally speaking, AI can also be broadly defined as the technique to manage uncertainty, “what you do when you do not know what to do”. Figure 2.8 shows a mindmap about possible sources of uncertainties. A complete scenario of a classical problem is defined in figure 2.9. The observable portion of the whole environment affecting the problems, as well as the actors, is the state. Metaphorically, the intelligent agents get the information with sensors and perform actions through actuators. These actions change the environment, not only the states. This is known as the perception-action cycle [93]. 2.4.1 Problems classification It is possible to classify the problems into several types regarding different considerations. For instance, the poker game would be defined as a partially observable, stochastic, discrete and adversarial problem. The different classifications are defined as follows: Fully vs Partially Observable In fully observable problems, it is possible to gather the necessary information to reach the optimal decision. This does not implies that the success of the agent only depends on 2.4 Intelligent Agents 25 its decisions, but it has everything it needs to decide optimally. For example, a card game with all cards up and uncovered. In partially observable problems, the agent has to make use of its memory to make the best decision. In this type of problems, anything which can be observed is considered to belong to the environment state. For example, poker. Deterministic vs Stochastic The concept of being deterministic or stochastic is similar to the classical meaning. A problem is considered to be deterministic when the resulting state after executing one action, or not executing anyone at all, depends only on the action itself. Stochastic problems are those where any movement or action can be conditioned to external agents, stakeholders or new variables which make impossible the exact and precise resulting state. This state does no only depend on the action of the agent. Discrete vs Continuous Discrete problems are those with a finite number of actions and a finite number of possible variables to consider before executing an action. The problems with a vast range of possible scenarios, such as the agents which play chess, are also considered finite. Continuous problems are those related to infinite possibilities for actions and states, such as autonomous driving or the optimization of energy market participations. Benign vs Adversarial In benign problems, the objective of the environment, which also includes opponents, has not any conflict with the objective of the agent. Hazard or stochastic problems can be examples of challenging benign problems. In adversarial problems, the environment (opponent) will try to defeat the agent. It is more difficult to make good decisions in this type of problems. 2.4.2 Neural Networks and Machine Learning Although NN and ML are not the only concepts in this field, they have become more and more important as the capacity of computation in the industry has increased. ML is strongly based on Bayesian Networks, presented in a Thomas Bayes’s posthumous publication in 1763 [94]. P(A|B) = P(B|A)P(A) P(B)(2.1) where: • P(A|B): likelihood of event A to occur, given that B ocurred (conditional probability). •P(B|A) is the conditional probability of B given A. •P(A) and P(B) are the probabilities of observing A and B respectively. In the 20th century, except for the AI winter from 1980s through the 2000s due to a lack of funding, ML has been widely studied and developed. However, the real rise was in the 26 Chapter 2. Literature Review 2010s, when Deep Learning (DL) became feasible. The requirements to train and develop NNs now are lower and it is possible for most of research centers to work with them. ML is mostly an empirical field. Theory remains behind the practice and research is getting more promising conclusions from experiments than from theoretical development. This is possible due to the large number of applications and frameworks which have been built over ML and to the huge amount of data which some datasets have stored. These fast improvements and discoveries, without a solid theory in the background, may lead us think about another AI winter [ 95 ]: a hibernation of the research in the field while theory is catching up the state of art of the empirical results. The most important features of NNs are the following [96]: • NNs can learn and model non-linear and complex relationships, which is very useful for real life problems. •They can generalize and infer unseen relationships on unseen data. • They do not impose any restrictions on the input variables, and they adapt better to heteroskedasticity, when the data present high volatility and non-constant variance. Universality in Neural Networks Universality defines a property which NNs present if they are able to approximate any continuous function. There are many attempts to prove the universality property of NNs from different perspectives. In [ 97 ], it is proved that multilayer perceptrons are universal assuming the use of sigmoid activation functions, which have been deprecated by the widespread use of Rectified Linear Unit (ReLU) ones. Nevertheless, the use of sigmoid functions is not necessary since [ 98 , 99 ] proved that only some type of activation functions make NNs not universal. Other examples of more complex architectures such as Convolutional Neural Networks (CNNs) [100] or Recurrent Neural Networks (RNNs) [101] have also been proved to be universal function approximators. 2.4.3 Neural Networks Classification There is a large number of architectures, and the state of the art is increasing extremely fast. Researchers and industry are continuously discovering new advanced architectures for specific use cases. In this subsection, most of the main accepted architectures for different purposes are defined. Simple Perceptron and Multilayer Perceptron Simple perceptron is the very first and most simple structure of NNs, which consists of two layers: input and output. The input layer receives the network inputs, performs the calculations and provides the result to the subsequent layers, as it is shown in figure 2.10. The output layer produces the final result according to the results of previous layers and other elements of the network definition, such as the activation function. 2.4 Intelligent Agents 27 Figure 2.10 Simple Perceptron. Simple perceptrons can be formulated as: f(x) =  n ∑ i xi∗wi+b(2.2) where x is a vector with input values, n is the length of the inputs, wi is the weight of each neuron and bthe bias of the function. Multilayer perceptrons are NNs with one or more hidden layers. The hidden layers are those internal and private layers which are not visible to the external systems, which increase the depth of the network. The deeper the NN is, the more versatile it is and the better approximations it is supposed to present. However, the Universal Approximation Theorem [ 102 ] states that any continuous function can be approximated with one single hidden layer. In addition, deeper NNs may overfit the dataset, resulting in worse general inferences than others with less deep agents. Figure 2.11 represents a sample of a multilayer perceptron agent. This figure shows a fully-connected network. Having all the neurons connected is not required, and the different ways of setting these connections for layers and neurons, in addition to other aspects such as the customization of the training process, will set the differences among the following more complex architectures. Convolutional Neural Networks CNNs are the state-of-art in applications such as voices interfaces, Natural Language Processing (NLP) and computer vision. They work better than fully-connected NNs with visual information due to their two main types of layers: convolutional and pooling layers. The convolutional layers, with a kernel covolving different subsections of the data, are used to extract the high-level features from the images. The layer applies a set of different filters which gives more depth to the data. The pooling layers are responsible for reducing the spatial size of the convolved feature. The main advantages of using these layers are the reduction of the computational power 28 Chapter 2. Literature Review Figure 2.11 Multilayer Perceptron. Figure 2.12 Sample of a deep CNN presented in [ 103 ] with an extremely good performance on image classification. required to process the data, as well as the extraction of the dominant features which are rotational and positional invariant. Thus, by combining layers of these two types, the network has deeper and deeper layers (but with reduced dimensions), so that the data can be analysed from the most specific features to the more general ones. In figure 2.13, a graphic example of the analysis of an image classification by a CNN is represented. In this type of NNs, the use of dropout layers [ 104 ] usually improves the final results. The main purpose of these layers is the reduction of the connections between layers and neurons when training. These connections have a probability of being retained during the training process. This increases the variability and reduces the complexity of the network since there are fewer connections after performing the dropout. In addition, they prevent the overfitting with the training data, so the network is able to better infere in more challeging situations. Figure 2.14 has two samples of networks with and without dropout. 2.4 Intelligent Agents 29 Figure 2.13 Graphical scheme of the transformation process of an image in a CNN. Figure 2.14 Standard NN (left) and NN after applying dropout (right), source [104]. Autoencoders Autoencoders are a special type of NNs whose output values are as similar as possible to the input ones. The hidden layers are narrower than the input and output layers, so the network must find a compressed version of the data, by actually learning the identity function [ 105 ]. Basically, the autoencoders reduce the data dimensions by ignoring the noise in the data. These networks are built with one or more hidden layers acting as the encoder, and other ones acting as a decoder. They are used with different purposes such as image noise reduction or data compression with loss. Figure 2.15 shows a schematized sample of a noise reductor system. Recurrent Neural Networks RNN are especially useful in problems with ordered data, such as NLP, machine translation, speech recognition or financial time series. Their output depends on the previous output and the current input values, establishing connections between neurons along a temporal sequence with a feedback loop [ 107 ]. Figure 2.16 shows a schematized version of the most simple version of a RNN. There are other more complex versions of RNNs, such as Long-Short Term Memory (LSTM) or GRU. 30 Chapter 2. Literature Review Figure 2.15 Noise reductor autoencoder sample with a sample of a number from MNIST database [106]. Figure 2.16 Schematized figure of a simple recurrent neural network. Long Short Term Memory Networks LSTMs is a classical type of RNN that can use their feedback connections to consider recent input events in form of activations [ 108 ]. They have two types of memory, one short-term memory for short-time patterns, and another one to understand the noisy and incomprehensible input sequences. This architecture improves some vanishing gradient problems which come with the standard RNN. LSTM layers are composed of one or more LSTM units. Figure 2.17 shows the architecture of a LSTM unit in detail. It can be splitted in different parts for a better understanding: • State (blue section): State of the cell without many interactions with the rest of elements. 2.4 Intelligent Agents 31 Figure 2.17 Long Short Term Memory Neural Network cell scheme. • Forget Gate (red section): This sigmoid layer decides which information is not useful anymore during the training process and removes it from the state. • Input Gate (orange section): It is composed of a sigmoid layer which determines the information to be added, and a tanh layer to create the new candidate to update the state. • Output Gate (green section): These sigmoid and tanh layers build the final output of the cell, which results in the input of the following cell in the network. In the scheme, Ct represents the value of the cell, ht defines the value of the hidden layer and xt is the input data, all of them at their corresponding sample times. The elements with a ’plus’ symbol perform bitwise plus operations, and the others with a ’cross’ symbol perform bitwise product operations. LSTMs have become very useful in time-series forecasting, speech processing or NLP as advanced RNN architectures. Gated Recurrent Unit Networks GRUs were first introduced in [ 109 ]. The cells in this architecture have two gates: the update and the reset gate. In the scheme, ht defines the value of the hidden layer, rt represents the value of the reset cell, zt defines the value of the update cell and xt is the input data, all of them at their corresponding sample times. The elements with a ’plus’ symbol perform bitwise plus operations, while the others with a filled circle perform the Hadamard product operation. Figure 2.18 shows the scheme of a GRU unit. The cell can be splitted in the following sections: •State (blue section): State of the cell. • Reset Gate (red section): Used to determine how much of the past information must be forgotten. 32 Chapter 2. Literature Review Figure 2.18 Gated Recurrent Unit cell scheme. Figure 2.19 Generative adversarial network scheme. • Update Gate (orange section): Used to determine how much of the past information is passed along to the future. •Final memory (green section): Section to calculate the outputs of the GRU cell. Generative Adversarial Networks Generative Adversarial Networkss (GANs) represent one of the most important improvements in the field of NNs. These networks are defined by a framework composed of two different NNs which are involved in an adversarial process, a zero-sum (minimax two-player) game where the gain of one agent is the loss of the other [ 110 , 111 ]. The two networks are in continuous improvement in a simultaneous training. Figure 2.19 shows a schematized version of a GAN. There is a generative model, which fakes data according to the original distribution; and a discriminative model, which estimates the probability that a sample comes from the training data rather than other from the generative model. Thus, the training procedure for the generative model is to maximize the probability of the discriminator making a mistake. 2.4 Intelligent Agents 33 Figure 2.20 Realistic artificial faces built by a generative adversarial network. The applications of GANs are infinite. GANs are able to perform image-to-image translations, such as the creation of very realistic images from sketches [ 112 ]. Furthermore, they can also be used for inspirational purposes in contexts where there is a strong need of creativity such as indoor design [ 113 ] or generating cartoon characters [ 114 ] with impressive results. In addition, they are very useful for these data-driven problems where the dataset is not very large, by increasing the number of samples to improve the training of the intelligent agent. In figure 2.20, it can be observed how well GANs work in the creation of realistic artificial faces [115]. Neural Networks in the scope of this dissertation The main objective of the AI agent in this dissertation is to mitigate the negative effects of the disturbances given by the forecast services. These disturbances imply uncertainty, which usually results in penalties during the operation stage since the optimization might not be accurate enough. This scenario is a clear example of the application of time series forecasting since the objective of the intelligent agent is to predict future disturbances based on the past behaviour. In the field of time series forecasting, the use of LSTM and GRU architectures are an increasing trend in the state of the art, as well as the use of encoder-decoder architectures. The intelligent agent is based on a encoder-decoder architecture composed of GRUs to get the advantages of both techniques and to improve the final results. The full definition of the NN can be found in section Neural Network Disturbance Mitigation Service. 40 Chapter 3. System model and Optimal Operation Strategy definition will be the DR solution, which has slight and optimal changes from the original one. 3.4 Profile Accuracy Profile Accuracy (PA) is a KPI defined to evaluate the PR execution. It is defined by: PA = t=24 ∑ t=1 kDA(t)−PR(t)k(3.4) This simplification allows comparison between different baselines and executions without having to consider the asymetric and non linear penalties that the difference between them could imply. That is, a measure of the difference between of the ideal profile generated during the optimization of the bids for DA and the real operation. 3.5 Intent Profiles In this section, the new concept of IP is introduced. It can be defined as specific strategies for operating simultaneous energy markets, according to the relation between potential penalties and potential risks of having baseline deviations [ 19 ]. For instance, the potential profit of the DA optimization implementing a Conservative IP when running the algorithms will not be as high as using a Risky one, but the overall system will be more resilient to forecast deviations. Thus, penalties will be lower. There is no discrete categorization of IPs, as the number of possible states in the state space is infinite. Defining the optimal IPs configuration is out of the scope of this dissertation, but it is one of the most interesting research and innovation lines to be accomplished in the near future. In the context of this dissertation, three different IPs have been defined according to the behavior patterns observed in the results presented in [ 17 ]: Risky IPs, Conservative IPs and Mixed IPs. The Mixed IP is developed by using the concept of DSV presented in section 3.6. Thus, SOs decide when to take risks or to be conservative by using different Mixed IPs, depending on their know-how. 3.6 Dynamic Storage Virtualization DSV empowers the use of storage units as energy buffers when operating simultaneous energy markets. The objective function of the optimization problem is to get the best performance for the VPP when participating as a single agent, which would result in certain losses for individual ENs. Thus, the business model must compensate the affected ENs with incentives to set the economic equilibrium among each VPP EN. Figure 3.3 depicts different IPs working with the concept of DSV. DSV enables the system not only to set different size storage allocations for each service, but also to set battery SOC s at certain levels at some time checkpoints according to the behavior of the grid. The system under consideration is similar to the one defined in 3.1, with a PV panel and a battery for energy storage. 3.6 Dynamic Storage Virtualization 41 Figure 3.3 Conservative IP (left) and Risky IP (right). Figure 3.4 Dynamic Battery Virtualization with Mixed IP. Let us consider the Mixed IP to be split into five different time regions as it is shown in figure 3.4: • Before 7 am. Grid behavior is very predictable because there is not any generation or load power. Real power setpoints must remain near to forecasted ones since there is less activity at homes during these night hours and only constant loads might be working (fridge, heating etc.). Risky IPs are recommended for this kind of time regions. • From 7 am to 10 pm there are three time regions when Conservative IPs are recommended. The first region, from 7 am to 9 am, presents a high risk of uncertainties due to the ordinary early home activity. This is the time when the end-users make their lives at home and may plug in some unexpected very energy-demanding appliances. During the second one, from 9 am to 4 pm, the forecasts for PV generation are very high, so any problem in the forecast results would imply large differences between real and forecasted values at execution time. The last region, from 4 pm to 10 pm, is similar to the one from 7 am to 9 am because the endusers arrive home and the demand profile becomes less predictable again. 42 Chapter 3. System model and Optimal Operation Strategy definition • After 10 pm, the time region is similar to the one before 7 am, so Risky IPs are recommended again. The hours limiting the different time regions are considered as soft checkpoints. These guidelines are not fixed and their only purpose is illustrative. They are meaningful in depicting the strategy which SOs can consider to operate the VPP, and how they mix the different IPs from a full-day perspective. Soft checkpoints are not stored nor processed in any algorithm execution. Knowing the environmental factors of the VPP, if possible, makes easier to set more accurate IPs. The previous Mixed IP is representative of a smart grid fully located in Borkum, Germany [17], but it may differ from other VPPs. The use of IPs makes the DSV independent of any algorithm execution and also decoupled from any energy service. Nevertheless, it is not recommendable to change the IP configuration after DA time without having a major reason, such as a massive breakdown or important issues with forecast service, so that it could be possible for the VPP to meet the commitment during the operation stage. Although the framework makes it possible, these advanced operational decisions are under SOs’ responsibility. 3.6.1 Individual or aggregated Intent Profiles IP can be defined individually for each EN, but they can also be defined at grid level (aggregated IPs). SOs can select the most appropriate mode depending on their business models, and how the end-users will be billed. In the context of this dissertation, the IPs have been defined at a grid level so that it would be easy to see the effect of applying the different strategies. 3.7 Framework Definition: A general overview The following subsections provide a high-level overview of the entire platform. Figure 3.5 depicts a full overview of the platform with all the integrations. The first step to start working with the platform is to select the target markets and the configuration of the grid. In this dissertation, the platform has been orientated to be used for DAM as well as to participate in DRP. The operation in multiple markets is achieved by developing a hierarchical multi-model which is composed of independent models that can be integrated with low decoupling. The DA optimization algorithm uses the intelligent forecast services to improve its accuracy when optimizing. The optimal result given by this algorithm is tracked by the PR algorithm at the operation stage. There is another algorithm integration with the tracking process: The results of the DR participation. This algorithm modifies the previous baseline so that the PR algorithm can track the new baseline and reduce the penalties in the new scenario. The inclusion of more energy market participations is beyond the scope of this dissertation, but it is possible as the platform is open to new integrations and enhancements. All the services have in common the intelligent use of the battery, as it is shown in section 3.6. Figure 3.6 defines a single execution of the algorithm for a DA participation. It details the layer of the operations before the gates closure, called optimization layer. The rest of 3.7 Framework Definition: A general overview 43 Figure 3.5 Platform General Overview. Figure 3.6 2-stage hierarchical framework execution example. the actions are executed at the operation stage, and they belong to the operational layer. In this dissertation, since DRP participation is also proposed and implemented, during the execution of the operational layer, different calls to the DR algorithm may occur, which will update the baseline according to new contract constraints, building the previously defined piped baseline. The SO can execute different simulations with several IPs, grid configurations or even considering different forecasting services. Nevertheless, this is only a help in the complex decision making process of bidding and, if the SOs have other considerations they can submit the profile according to their needs. 4 Optimal Virtual Power Plant multi-service participation Technology has really enabled the sharing economy to not only become a major trend, but (also to) continue to grow and be successful Leah Busque Th e multi-service proposed in this dissertation is composed of a DA individual bidding optimization with participation in DRP. The strategy consists of two stages. The first stage optimizes the strategy for bidding in DA. The second stage consists of a control strategy to mitigate deviations and potential penalties (PR layer). DRP integration takes place during PR execution and modifies the proposed baseline in DA by the use of the piped baseline concept (see section 3.3). 4.1 Bidding Optimization for Day-Ahead Market Participation The first stage is defined by an optimization algorithm that generates the optimal baseline. Figure 4.1 depicts a sample of a baseline. Positive values define sample times when the VPP is supposed to inject power into the network, and negative values when the VPP is supposed to consume. The optimization problem is formulated as a MILP based on MPC. This algorithm receives the forecast of energy prices (injection and consumption) and the forecasts of every EN load and renewable source generation. The model also receives the grid topology as well as the physical limitations of every EN, such as storage capacity, charge / discharge efficiencies, etc. The specification of these parameters can be found in section 3.1 in detail. 45 46 Chapter 4. Optimal Virtual Power Plant multi-service participation Figure 4.1 Example of an optimal grid profile (baseline) for a day with good generation and a low energy-demanding VPP. 4.1.1 Mixed Logic Dynamic Systems MLD systems were introduced in the literature as a framework for modeling and controlling systems described by interdependent physical laws, logic rules, and operating constraints. These are described by linear dynamic equations subject to linear inequalities involving real and integer variables [ 117 ]. In this dissertation, the MLD system also includes boolean variables for solving problems with non-linerarity. In [ 117 ] a generalized MLD definition through the following linear relations is presented as: x(t+1) = Atx(t)+B1tu(t)+B2tδ(t)+B3tz(t)(4.1) y(t) = Ctx(t)+D1tu(t)+D2tδ(t)+D3tz(t)(4.2) E2tδ(t)+E3tz(t)≤E1tu(t)+E4tx(t)+E5t(4.3) where x is the state of the system, y is the output vector, u is the command input (which can be continuos or binary on/off commands), δ∈0,1 and z∈R represent respectively auxiliary logical and continuous variables. MLD systems are solved through MILP or a MIQP, depending on the type of objective function used in the algorithm. In this dissertation, MILP has been used to optimize bids in DA and DR. A MIQP implementation has been chosen to solve the PR optimization. 4.1 Bidding Optimization for Day-Ahead Market Participation 47 Figure 4.2 SO types. On the left, an energy producer SO; on the right, a market participantproducer SO. 4.1.2 Optimization Scenarios The framework considers four different scenarios for optimizing the VPP participation in DAM depending on network costs and the type of SO [118]. •Aggregated Billing, with P2P. •Aggregated Billing, without P2P. •Individual Billing, with P2P. •Individual Billing, without P2P. Individual billing is formulated for SOs playing the role of energy-producer, whereas aggregated billing is defined for those working as market participant-producer. Figure 4.2 depicts the different types of scenarios. At the operational level, the main difference between the two modes is that the former requires the forecasting of consumption and injection prices for every node, and the latter only requires the forecasting of the prices of the VPP. In fact, individual billing option makes more flexible business models possible with custom agreements for each client. The framework enables SOs to set a P2P cost to consider some network transmission costs in the optimization too. This cost will reduce transmission actions between nodes and reinforce the operations with the external network. P2P prices must also be provided individually, or for the grid, depending on the mode. Energy transmission always implies a cost, so P2P modes enable more accurate optimizations for those cases when the SOs know the real cost or when they have a good enough estimation. 4.1.3 Algorithm inputs The algorithm receives different inputs depending on the mode to be executed: 48 Chapter 4. Optimal Virtual Power Plant multi-service participation • 24-hour injection and consumption market price forecast. These prices are provided from external forecast services for every EN or for the whole grid depending on the optimization mode. • 24-hour P2P forecast, which is optional depending on the optimization mode. It can be constant, but also specified for every hour and every EN. •24-hour load and generation power forecast of every EN. • Grid topology in terms of physical limitations of every EN, such as storage capacity, charge / discharge efficiency, batteries maximum charge / discharge power and the limitation of the nodes and grid in terms of the maximum allowed power in the connection point to the network. 4.1.4 Simulations and submissions DA algorithm is prepared to be run as many times as needed to create different profile optimizations. The SO can perform the optimizations with different topologies of the grid such as including or excluding specific nodes. They can even optimize with different available IPs to extract the necessary conclusions. The results are not definite so that the SO might be able to build their desired profile, which will be used as the baseline to be tracked by the PR algorithm. Some systems are more stable and predictable than others with respect to their future behaviour. In these cases, the know-how of the SO could be expendable and the DA requests can be configured with the option of autosubmit so that the solution of the DA algorithm will be automatically registered as the optimal baseline for the operation stage. 4.1.5 Objective and Cost function The objective of the problem is to optimize the aggregated Market Settlement (MS) for all the ENs during a full day. Considering this, the objective function can be defined as: max i=tend ∑ i=1 MS(ti)(4.4) where tend is the last operation hour. This value can differ from 24, depending on daylight saving events. For the purpose of this paper, tend = 24. The MS is defined in this context by the revenue of selling energy to the network and the cost of consumption. Considering this, MS is given by: MS(t) = N ∑ k=1 MSk(t)(4.5) MSk(t) =                (Pricein j,k(t)−PP2Pk(t)) ∗Pex,k(t) for Pex,k(t)>=0 (−Pricecon,k(t)−PP2Pk(t)) ∗Pex,k(t) for Pex,k(t)<0 (4.6) 4.1 Bidding Optimization for Day-Ahead Market Participation 49 Most of the times, tend will be 24 because the optimization is supposed to be executed for a full day. However, in most of the countries there are two special days every year when daylight saving events happen. In these days, the country changes its time zone so these days can have some more, or less, hours, depending if the event sets the clock back or forward. In other words, in most of the cases, there is one 23-hour day in late winter (or early spring) and one 25-hour day in autumn. Thus, rigid implementations would make the system out of order twice a year. The proposed platform considers this issue and it is stable and robust even for these corner cases. Special days are dynamically calculated so that the validations will not fail and it will be checked if the algorithm call is done with the appropriate number of values in these special days, and 24 values for ordinary ones. 4.1.6 Linear State Space Model The dynamic of the state variables is given by the equation 4.7: SOCk(t+1) = SOCk(t)+ Pcharge,k(t)∗ηcharge,k∗Ts∗100 Capk +Pdischarge,k(t)∗Ts∗100 ηdischarge,k∗Capk (4.7) where SOCk(t) represents the state of charge in percentage; Pcharge,k(t) and ηdischarge,k are the power charge and discharge respectively given a node k ; Ts defines the sample time; Capk the capacity of the battery; ηcharge,k and ηdischarge,k are the efficiencies given a node with values from 0 to 1, where 1 is the ideal scenario. 4.1.7 Model Specification The receding horizon of the MPC model is 24, with a sample time of 1h. The implementation of the algorithm is prepared to use a different horizon considering daylight saving events, depending on the event (start or end of the daylight saving time) and the timezone. The variables are built dynamically depending on the number of ENs. They can be grouped and defined as: •PDA(1)...PDA(nh)- Continuous variables for aggregated power •Pbat,1(1)...Pbat,1(t),Pbat,k(t)(1)...Pbat,K(nh) - Continuous variable for each EN. One per sample time and EN. •Profit. Continuous variable to optimize the profit in the formulation •L1(1)...L1(nh) Continuous variable to linearize the problem, one per sample time. •L2(1)...L2(nh) Integer (binary) variable to linearize the problem, one per sample time 56 Chapter 4. Optimal Virtual Power Plant multi-service participation Figure 4.9 SOC evolution for a Risky IP execution. Figure 4.10 SOC evolution for a Conservative IP execution. these 18 setpoints, the ENs can work correctly even with network failures until entering standalone mode. In addition, the PR algorithm can deal with these offline nodes as if they were working as expected, so it is possible to maintain them in the pool. There are four different EN statuses that can be managed by the system, as it is described in subsection 3.1.3. Most of the issues which may appear during the control stage are related to data, since the platform is executed remotely and any problem gathering every node SOC , setpoints and short-term forecasts will imply wrong results of a PR execution. 4.2 Penalty Reduction Optimization 57 Figure 4.11 SOC evolution for a Mixed IP execution. 4.2.1 Objective and cost functions The function to be optimized is the difference between the DA profile and the new schedule. Some deviations may occur due to the unavoidable uncertainties in this kind of problems, such as a low accuracy in forecasts for generation or load profiles or the appearance of unexpected malfunction errors. However, the platform is designed to mitigate most of these uncertainties by the use of DSV and IPs. Given this, the objective function is as follows: min ti+18 ∑ ti DA(t)−PR(t)(4.16) where DA(t) is the baseline set by the DA algorithm and the PR(t) is the summatory of each individual EN setpoints. In this stage, the sample time is 10-minute, not for one hour as it was in the previous stage. All the setpoints of generation and load for EN correspond to a short-term forecast. These values will be more accurate than the ones obtained during bid time the day before. These forecast values must be provided at every PR step so that they could be refreshed to improve the control performance. 4.2.2 System Constraints The optimization problem is subject to the system constraints, defined as follows: Grid constraints These constraints define the power flow equations, enabling the aggregated optimization of all EN participation at every sample time to reduce the deviations related to the current 58 Chapter 4. Optimal Virtual Power Plant multi-service participation baseline in execution. PGRID(t) = K ∑ k=1 Pex,k(t)+ K ∑ k=1 Pgen,k(t)− K ∑ k=1 Pload,k(t)−PBase(t)(4.17) where Pex,k(t)is the setpoint of the node kat time t. The baseline can be modified by DR calls, as it is described in 4.3. In that case, the constraint is slightly changed since the baseline to meet is not the DA one, but the last DR baseline result. The constraint 4.17 is a generalization of 4.18. PGRID(t) = K ∑ k=1 Pex,k(t)+ K ∑ k=1 Pgen,k(t) − K ∑ k=1 Pload,k(t)−PDRn(t) (4.18) Energy node constraints The first two equations define the exchange limits for every node. The following two constraints establish the unavoidable physical limits related to the battery. Finally, the last two ones determine EN SOCs in different optimization sample times. Pex,k(t)⩽Conmax,k−Pgen,k(t)+Pload,k(t)(4.19) Pex,k(t)⩾Conmin,k−Pgen,k(t) +Pload,k(t)(4.20) Pex,k(t)⩽Pbat-max-discharge,k(4.21) Pex,k(t)⩾−Pbat-max-charge,k(4.22) SOCk(t)⩽SOCmax,k(t)−SOCk(t−1)(4.23) SOCk(t)⩾SOCmin,k(t)−SOCk(t−1)(4.24) 4.2.3 Fault Detection Enhancement Several strategies have been designed and implemented to enhance the system robustness by considering some fault detection problems: the intrahour equilibrium and the management of non controllable nodes. The former improves these scenarios where there are deviations in time gaps shorter than one hour, which would result in deviations of the mean power supplied, or consumed, for this given hour. The latter maintains the ENs with read only status in the pool of available nodes in case it would be interesting for the operation. 4.2 Penalty Reduction Optimization 59 Intrahour equilibrium strategy In this dissertation, intrahour equilibrium strategy refers to a strategy to solve issues related to the multi-time intrinsic feature by interpolating deviations for intervals of time of less than one hour (which is the DA sample time unit) and, consequently, minimizing the impact in the following executions. There are two ways to perform the intrahour equilibrium: the average power mode and the energy mode. At every sample time in average power mode, the PR algorithm calculates the accumulated average power which the VPP has operated during the current hour. If the average power differs from the commitment in DA, all the following 10-minute sample times within the same hour must compensate for this deviation as much as possible. Due to the fact that the deviation penalties, when the VPP is supposed to inject energy into the power system, are much higher than the ones in consumption, the intrahour equilibrium is only activated for sample times when the commitment in DA is for injecting power. Pre f (ti) =                        DA(t)−∑i−1 j=0∑K k=1Preal,k(tj)∗Ts 1 Ts −i for DA(t)>0 DA(t) for DA(t)≤0 (4.25) where i is defined in the natural interval of 0-5 since Ts is defined every 10 minutes. The number of available sample time depends on the PR algorithm execution time. Figure 4.12 shows a full-day simulation with deviations. The series shown in blue color represents the average power for the VPP to exchange during the same hour, but it can be observed in red color that the real setpoints, which were actually commanded to the EN, are slightly different. The shape of the chart representing this second series is characteristic of a system which has lost its generation source. Consequently, the optimization algorithm will try to reduce the deviations without success since errors do not converge. Two different models have been implemented for running this enhancement, depending on the SO needs. • Energy Mode: The SO indicates the amount of energy that the grid has injected or consumed from the network during the current hour in the inputs of the PR algorithm. This mode is very conveninent for smart grids with ENs composed of a smart-meter that can be queried in real time. • Power Mode: It is activated by default. The system tracks the real power executed by each EN and creates a new virtual baseline for the current hour to be optimized in the current execution. • Off: The intrahour enhancement can be completely deactivated. It is not recommended, unless the SO has strong knowledge about the actions which they are 60 Chapter 4. Optimal Virtual Power Plant multi-service participation Figure 4.12 Comparison between power mean and real setpoints. performing. In most of the scenarios, having this enhancement activated will improve the overall optimization. In section 4.2.3 there is a case study to illustrate its importance. It is important to remark the strong dependency that exists when the power mode is activated. In this framework, every algorithm remains absolutely decoupled from others, being coordinated by the powerful role of the storage system. This means that not only every execution of the algorithms are independent, but also several steps of the same algorithm will not present any dependency from others. Having power mode activated implies that the virtual baseline will depend on previous executions, and it would not be possible to run algorithms in the real environment without affecting the following steps. This is not an actual problem for operation since each PR execution will be executed only once in a real environment. To solve this issue, it can be deployed other entrypoint in the Application Programming Interface (API) of the system only for simulation purposes with an implementation which does not interfere with the operation of the smart grid. There is more information about the software architecture and the implication of new developments in chapter 7. Intrahour enhancement can be activated or deactivated in every sample time by setting the request. By default, power mode is enabled since this is the best option for most of the times, according to the research. However, the platform gives the operator the freedom to switch it on, or off, depending on their context. Non controllable nodes There are some occasions when the ENs are not controllable although they are able to send the telemetry (read-only status, see 3.1.3). However, it would be profitable to maintain these nodes even without controlling them, but adapting the VPP to compensate their behavior. Two clarifying scenarios could be: 4.2 Penalty Reduction Optimization 61 • The read-only EN is generating much energy and its load profile is not very demanding. The excess energy can be saved by other EN. •There is a DR action and the VPP needs the inclusion of more ENs. The model to adapt the VPP to read-only nodes is defined as: Pbattotal ,k(t) = Pbatbat ,k(t)+Pbatnetwork,k(t)(4.26) Pbattotal ,k=FSk−Pgen,k(t) +Pload,k(t)(4.27) where FSkis the observed fixed setpoint due to failures of the node. SOC(t) =                              SOC(t−1)−Pbat ∗Ts for SOCmin <SOC(t)<SOCmax SOCmin for SOC(t)⩽SOCmin SOCmax for SOC(t)⩾SOCmin (4.28) Pbatbat ,k(t) =                                      −(SOC(t−1)−SOC(t)) Ts for Pbat-max-charge,k <Pbatbat ,k(t) <Pbat-max-discharge,k Pbat-max-charge,k for Pbatbat ,k(t)⩽Pbat-max-charge,k Pbat-max-discharge,k for Pbatbat ,k(t)⩾Pbat-max-discharge,k (4.29) Considering that the real power exchange is the difference between the fixed reference and the power battery that applies to network, it can be concluded that: RSk(t) = FSk−Pbatnetwork,k(t)(4.30) Substituting with 4.26: RSk(t) = FSk−(Pbat−total(t)−Pbatbat ,k(t)) (4.31) The value of RSk(t) is the real setpoint that the read-only EN is actually performing. PR algorithm considers this value and it adapts the setpoints of the other nodes so that the aggregation could adapt the operation. 62 Chapter 4. Optimal Virtual Power Plant multi-service participation Figure 4.13 Complete day simulation without generation in any node. Meeting injection It is extremely important to consider that deviation penalties when the VPP is supposed to inject power are much higher than the ones in consumption [ 121 ]. PR algorithm is able to consider more deviations in consumption sample times to meet as much as possible during injection ones. This is possible because of the intrinsic receding horizon of the MPC formulation. The main type of actions which can be commanded are two: • Inject-to-consume: Actions needed when the grid has more stored energy than optimal. It could be anti-intuitive to assimilate the fact that having an excess of clean renewable energy could imply problems, but there are some systems which cannot drop energy. Thus, they would not be able to store the amount of energy needed resulting in additional unexpected energy injection actions. • Consume-to-inject: The energy is bought before-hand in sample times when the grid is not injecting to meet the commitment in following injection sample times. Figure 4.13 depicts a complete-day execution where the real values for generation have been submitted as 0 during operation time, instead of the forecasted values. Due to this scenario, the batteries get empty, so meeting high injection sample times remains complicated. Physical limits of ENs and batteries make impossible faster charge actions to meet the injection profile. However, there can be found many sample times with consume- to-inject actions located before every injection peak. In these sample times, the VPP consumes more energy than needed to store it and use it for injection. The area delimited by the red slashed line is an example of consume-to-inject action. The green one is the injection peak which has been successfully satisfied. 4.3 Demand Response Program Integration 63 Figure 4.14 Execution with Intrahour Equilibrium Off. The importance of the injection meeting enhancement during operation The design and the implementation of this fault detection enhancement is extremely important. It can be observed in 4.14 and 4.15 the difference between the accuracy of the two simulations. In this simulation, the forecast values for generation and load are set without any disturbance in relation with the real ones during the control stage, except for the steps 42 and 43. In these steps, the real generation was 5kW higher than forecasted for each node. The grid is not able to compensate these deviations during the following four sample times that still belong to the same hour. Thus, the injected energy in the grid for this sample time (equivalent to the mean of the power since the tracking sample time is of one hour) is much higher. The PA is 2.77 with the enhancement activated and 23.45 otherwise. Then, the grid without the enhancement activated presents 88.2% less deviations than the other in terms of absolute deviations considering this case study. This is possible due to the fact that the grid was able to absorb the impact of the deviations during its control stage. As it can be observed, the deviations happened during only one hour have a big impact for the rest of the day, which means that not having the optimal SOC in each node compromises the contract for the rest of the day. 4.3 Demand Response Program Integration In this section a DR model is presented based on MPC and MILP. This model is integrated with the reconfigurable multilevel dynamic model for the microgrid optimal operation. DR adds more flexibility to the decoupled system since it generates a new profile which overrides the scheduled actions for the nodes in the grid, in the interval of time between a DR execution and the end of the day. This last profile is considered to be the active one, 64 Chapter 4. Optimal Virtual Power Plant multi-service participation Figure 4.15 Execution with Intrahour Equilibrium On. while the previous ones are all deactivated. The resulting grid profile which is operated at the end of the day will be the conjunction of all the active ones, which was defined as piped-baseline in 3.3. The algorithm is designed to handle real operation simulations so that the operator could be able to simulate different scenarios to assess how the grid would perform after every optimization. Once the DR action is approved, the profile must be explicitly submitted to the multilevel dynamic model. The DR algorithm has been designed as an expansion of DA formulation, but absolutely decoupled and independent. SOs can run this algorithm every time that the market makes an offer for an increase or reduction of the consumption from the network. This optimization needs the updated forecast measures and the current VPP state (ENs availability, individual SOC s, etc.). The VPP will try to satisfy the new scenario by modifying the commands of the following hours which were not scheduled to inject, avoiding the impact on injection sample times so that high penalties would be reduced. If it is possible for the VPP to allocate the changes, the algorithm will generate and persist the new baseline in the database, or no solution otherwise. Only if the injection profile is not compromised, DR applies. 4.3.1 Objective and Cost function The objective function is defined as: max tend ∑ tDRn MS(t)(4.32) where MS is similar to the equation defined in 4.5, but only with the sample times that are between the time of execution and the end of the day. The DR service can be called with different intervals depending on the TSO’s needs. An interval is defined as: 4.3 Demand Response Program Integration 65 •Init hour. •End hour. •Incentive price. •Load offset (positive or negative) to be reduced or incresed during interval time. 4.3.2 Specific System Constraints Profile limits In order to ensure the optimal performance by reducing penalties, it is necessary for the aggregation to maintain the values of the won bids on the day before, and to limit conversions. A conversion is defined as the change from consumption mode to injection mode, given a sample time. The only allowed conversion is during the DR interval and only if feeding compensation is enabled, as it is explained in the next subsection. Thus, PDRn(t) = PGRID(t)∀t∈PGRID(t)≥0(4.33) PDRn(t)≤0∀t∈PGRID(t)≤0(4.34) Feeding compensation vs No-Feeding compensation Feeding compensation occurs when the DR optimization changes the value of a previous established command for one hour from a negative value to a positive value. This means a change from a consumption command to an injection one. This should be forbidden since the bid was not won for this sample time during the day before, so the VPP would not be allowed to inject. Nevertheless, this flexibility enables new economic operations and agreements between the TSO and the SO, as well as potential benefits for prosumers. Disabling feeding compensation constraints are given by: PDRn(t) = max(0,PGRID(t))∀t∈ {tDRn. . . 24}(4.35) where t is an array with 24 −t values; PGRID(t) , the aggregated setpoint of the grid at time t ; tDRn the sample time of the nth execution of the DR algorithm; PDRn(t) , the aggregated setpoint of the pipe baseline at time t. 4.3.3 Fast Demand Response (Maneuvers) DR algorithm integrates a system of Fast Demand Response (FDR) to participate in the DRP under the SO’s criteria. The system generates one maneuver for consumption increase and other manuever for consumption reduction, for each of the following two hours, at every PR step (every 10 minutes). This value depends on three factors: the capacity of 72 Chapter 5. Chance Constraint Optimization where Fst (t) is the stochastic factor, the result of the DML at a certain sample time. This term is defined as: Fst (t) = fc1ϕ−1 genk,t(δ),ψgenk(t)−fc2ϕ−1 loadk,t(δ),ψloadk(t)(5.3) F+ st (t) =        Fst (t)for Fst (t)≥0 0otherwise (5.4) F− st (t) =        Fst (t)for Fst (t)≤0 0otherwise (5.5) where fc1 and fc2 are functions to combine the results, depending on some selected criteria; ϕ−1 genk,t(δ) and ϕ−1 genk,t(δ) are the results of applying the CC models for PV and load respectively for each EN and sample time, with a risk factor of (1−δ) ; ψgenk and ψloadk define the NN models for a given EN. fc1 and fc2 have been defined in this case study as a simple linear combination of both services. The definition of more complex combination functions is possible, but out of the scope of this paper. The following subsections describe the implementation of each component in the stochastic factor. Stochastic Component Service The stochastic component has been developed as a customizable optimization feature which can be deactivated on demand at each optimization run. In the implementation, the stochastic factor is given by an isolated service which obtains both the CC and the NN method and calculates the result by a simple linear combination of both ( fc1 and fc2 ). The combination of each participant is under the SOs’ criteria. The pseudocode in the algorithm 1 outlines the implementation of how the stochastic component service obtains the stochastic factor. 5.1.2 Chance Constrained Disturbance Mitigation Service CC are used to solve problems where uncertainties may exist and the constraints have to be satisfied in the vast majority of cases, but not necessarily in all of them. This can be formally written as: minimize f(x,ξ) subject to P(g(x,ξ)≤0)≥1−ε where Pis the probability of the expectation so as not to violate these constraints. According to [ 89 ], the information about the prediction error can be modeled by the Probability Distribution Function (PDF) by comparing historical data with their respective predictions. The Cumulative Distribution Function (CDF) can be deduced from the 5.1 Disturbance Mitigation Layer 73 Algorithm 1 Calculate Stochastic Component δ←0.95 DCC gen ←/0,DCC load ←/0 DNN gen ←/0,DNN load ←/0 nodes ←EN1..ENk for n in nodes do for t in [1..24] do DCC gen ←append(DCC gen,ϕ−1 genk,t(δ)) DCC load ←append(DCC load,ϕ−1 loadk,t(δ)) end for DNN gen ←ψgenk(t) DNN load ←ψloadk(t) Fgen ←fc1(DCC gen,DNN gen) Fload ←fc2(DCC load,DNN load) end for return Fload(t)−Fgen(t)∀t∈[1..24] where DCC gen , DCC load are two lists of results from the CC algorithm (with 24 values each, one solution per sampling time), with a risk of violation of the constraint of (1−δ=0.05). DNN gen and DNN load are the results (24 values each) of the ED model, Fgen and Fload the result of combining both aproaches. previous comparison to describe the cumulative distribution of the disturbance. This function can generally be defined as: FX(x) = P(X≤x),∀x∈R(5.6) The stochastic factor in the MPC is included in the formulation by combining the CC and the NN components. The steps to build the CC component are defined in the following subsections. The approach to collect the data to build the CDF will depend on the nature of the system to be operated. In the case study presented, the uncertainties in the generation forecast for PV panels are very different depending on the hourly correlation with respect to the working day. Forecast systems are less likely to present deviations in the intervals of time when there is no solar radiation than in others in the central hours of the day. Consequently, in order to build better solutions, it is necessary to create 24 different models per EN. The data to be gathered are the historical deviations between forecasts and actual values per EN. In case that the platform has different sources of forecast, it is also possible to indicate which the source is and to build different models for each one. Thus, one model is built for each tuple of forecast source, EN and sample time. The size of the data is not important in terms of performance since the models are built and stored only once. It is possible to rebuild these models according to the SO’s know-how since the source of the forecast service may have changed, or the VPP may 74 Chapter 5. Chance Constraint Optimization present some new developments from a specific date, which could affect the accuracy of the following forecasts. Cumulative Distribution Function Definition and implementation A CDF function can be defined by giving values within the domain of the model function in systems where the model can be formulated and is well known. Consequently, when defining the CC model, CDF−1can be calculated, if it exists. However, when working with real systems in real environments, the model to be used is a data-driven one. In fact, this is the real problem in this context, when the disturbance model is very hard to formulate by means of classical methods, and the system tries to compensate the disturbances with additional actions. Consequently, an Empirical Cumulative Distribution Function (ECDF) is necessary to be formulated based on historical data to build the model with its inverse. The only difference between this function for PV forecast, and the implementation of the CC model for loads, is the predictor component. The predictor is the agent which fetches the disturbances from the database. The implementation of ECDF−1 must interpolate the results obtained from the extrapolation of the ECDF(δ) resolution for each historical data saved at the given sampling time. After building the models, all of them must be executed with the appropriate risk of violation of the constraints. In this dissertation, (1−δ) = 0.95 . In the implementation, pv_pred is the disturbance predictor of PV. Code 5.1 Implementation of CC model. from scipy.interpolate import interp1d from statsmodels.distributions.empirical_distribution \ import ECDF def build_pv_cc_model(en, t, fc='default'): model = lambda x: x disturbances = set(pv_pred.get_all(en, t, fc)) if not len(disturbances): return lambda x: 0 elif len(disturbances) == 1: return lambda x: disturbances[0] else: ecdf = ECDF(disturbances) keys = sorted(disturbances) return interp1d([ecdf(v) for v in keys], keys, fill_value="extrapolate") The implementation of ECDF−1must interpolate the results obtained from the extrapolation of the ECDF(δ) resolution for each historical data saved at the given sampling time. After building the models, all of them must be executed with the appropriate risk of violation of the constraints. In this paper, (1−δ) = 0.95. 5.1 Disturbance Mitigation Layer 75 Figure 5.1 Encoder-Decoder Architecture. 5.1.3 Neural Network Disturbance Mitigation Service In recent times, the number of possible applications for NNs and their impact on society has been increasing in fields such as classification, clustering, pattern recognition and prediction. In the field of time series, the current state of the art shows that RNNs present better results than traditional NNs. RNNs can be built by two main approaches: LSTMs or GRUs. It is possible to establish forward and backward dependencies between RNN layers by means of building Bidirectional Recurrent Neural Networks (BRNNs). Although the backward dependency may seem not to make sense in forward predictions, this combined strategy is widely being used successfully in NLP. Finally, one of the most accepted aproaches are attention layers. This architecture consists of an encoder-decoder which sets not only dependencies between the different features, but also between features of different sample times. In this dissertation, an ED architecture is defined and implemented. The encoder is defined with an unidirectional GRU layer and the decoder cell are defined as GRU cells. Neural Network Model Definition The definition of a GRU cell can be found in [ 139 ]. Figure 5.1 shows the full architecture of the NN. The inputs are defined with two different types of data: • Past Sequential data: Last 24 values of disturbances in the forecast service and other step-dependent features such as the daily correlation or weather factor. • Main Features: Features which remain the same for the whole prediction: categorical data, averages and EN identifier. Logit functions are those which perform logistic regression with outputs from 0 to 1 (regression probability). These functions are widely used in the last layer of NNs and they take as inputs the outputs of other deeper layers in the network for use cases such as pattern detection or information classification, for example. The output of the encoder is the result of its last hidden layer without applying any logit function. Thus, the decoder 76 Chapter 5. Chance Constraint Optimization Table 5.1 Encoder-Decoder Hyperparameters. Encoder - Decoder Optimizer AdamW Learning Rate 0.001 Loss Function MSE Sequence Length 72 Input Feature Length 5 (one of them with 24 values) Hidden Size 300 Output Size 24 receives the raw values of this hidden layer, as well as the previous and future features. The output of this layer is the sequence of 24 values with the forecasted disturbance error to fix the original forecast profile. There are other settings as important as the architecture or the data when working with NNs: hyperparameters, optimizers, etc. In the proposed case study, table 5.1 defines the best settings found for the NN after iterating over different configurations. The hyperparameter that had the most important impact on the accuracy of the network in this case study was the optimizer. Tests showed that AdamW [ 140 ] was the best optimizer, presenting better performance in forecasts since it reduces the overfitting. It is an improved implementation of Adam optimizer. 5.2 Results The prices of the imbalances are traditionally calculated by considering different complex criteria depending on the state of the power system [ 141 , 142 , 143 ]. In addition, power systems can be split in different zones and each one can have its own approach to penalize the imbalances. However, there is a clear pattern: the incentives to respond to the demands of energy markets to fix deviations, as well as the penalties for not meeting the commitment, are always higher than regular prices, as it can be observed in different zones such as Belgium [ 144 ] or the Nord Pool in the North of Europe [ 145 ]. In this paper, the assessment of the results will be presented considering that the price per kWh in penalties is higher than regular prices, and the evaluation will be performed in terms of relative savings and improvements. The forecasts can be divided into two main categories: pessimistic and optimistic. Pessimistic forecasts are those which generally predict a lower power generation, or a higher power load, than real values. Optimistic forecasts are just the opposite. Figure 5.2 depicts a sample of each type of forecast. Most of the time, the forecast might not show any clear trend but, precisely in these cases, the effects of the disturbances are less critical for the optimization. They are still important, but some sample times with overestimations can, to some extent, compensate for other understimated forecasts. Optimistic forecasts have the advantage of generating 5.2 Results 77 Figure 5.2 Pessimistic generation forecasting (red) vs Optimistic generation forecasting (green) vs Real generation values. more profitable DA bids, but a worse performance in the operation and, consequently, more potential penalties. Pessimistic ones have the disadvantage of generating less profitable DA bids, but also present a worse operation performance. In this case, ENs are supposed to generate less energy, so the grid is prepared to handle this volume of expected energy. Consequently, the batteries can be filled up, causing penalties due to the deviations. 5.2.1 Disturbance Predictors The CC disturbance predictor is built with the models of each EN and sample time, precalculated and cached in-memory to be executed faster. The 24 results of each node are combined with those obtained from the ED disturbance predictor. The ED disturbance predictor receives the last 24 values of the disturbances of a node, as well as some pre-processed features related to the day of the execution and the weather. The data to train the NN has been downloaded from the London Datastore Repository [ 146 ], and processed to add some weather criteria to build the disturbances. This dataset contains data collected from 20 substations and 10 domestic premises over 480 days with a sample time of 10 minutes. Figure 5.3 shows a sample of a disturbance forecast run with both methods. 5.2.2 Case Study: DML Performance Assessment The best possible scenario to use the DML is with the mitigation of deviations of pessimistic forecasts. The optimization will be more profitable than in the optimistic forecasts since the real values, in generation or load, will be lower than expected, so the variable defined to optimize the revenues can be maximized with a higher value. In addition, penalties will be lower due to the fact that the fixed forecasts are more accurate than the other ones with errors. 78 Chapter 5. Chance Constraint Optimization Figure 5.3 Sample of disturbance forecasting with CC and ED. The main objective of the DML is to reduce the penalties without a dramatic impact in the revenue. The market settlement may be slightly lower at the time of optimization, so the optimization might seem to be more profitable without the activation of the DML. However, the problem arises when the grid is being controlled at operation time, since the penalties also turn to be much higher. The penalty reduction algorithm implemented to control the grid considers that the penalties for not meeting the commitment at the time of injection are much higher than those applied during the consumption sample times. The system is capable of fixing future deviations in injection sample times by modifying its aggregate consumption profile. It is achieved by performing consume-to-inject actions up to 3 hours after the last operation time, which is the time interval of the receding horizon. This asymmetrical control technique improves the accuracy of the profile in injection times compared to consumption ones [17]. In table 5.2, the assessment of several 15-day simulations with full-day operations in the VPP is presented, having the DML deactivated (deterministic model) or with partial or total layer activation. Real and optimized profiles are compared by their Mean Squared Error (MSE). All the studied scenarios have been run with the same forecasting error, storage and load conditions. The control algorithm has the same configuration and implementation. The only difference between the optimizations is the activation of the DML, or the execution of the optimization with the DML deactivated. The columns with ∆ values represent the average of the relative value of MSE with respect to the deterministic approach for every day, according to the following equation: ∆= 15 ∑ d=1 Stochastic(d) Deterministic(d)−1!∗100 (5.7) 5.2 Results 79 Table 5.2 Different executions with the complete/partial activation, or deactivation, of the DML. DML MSE MSEin j MSEcon ∆mse ∆in j ∆con Deactivated (deterministic) 955.82 391.39 564.43 - - - Fully activated (stochastic) 933.25 381.73 551.52 -11.85% -11.10% -7.93% Only CC activated (stochastic) 938.14 384.34 553.80 -9.67% -8.92% -7.42% Only NN activated (stochastic) 955.82 391.39 564.43 -11.92% -7.61% -8.42% Figure 5.4 Heatmap with the cardinal differences between the MSE with deterministic and stochastic versions of the problem. The top horizontal bar shows the average improvement for the whole day. where opt corresponds to the three different analysed stochastic optimizations: CC , NN , or combined. In figure 5.4, a heatmap showing the differences between the deterministic execution and the stochastic one is shown. It can be observed how the central hours are more sensitive to uncertainties since solar panels focus their activity in the time interval from 12 pm to 16 pm approximately, so PV forecast generation may be less accurate. In addition, pessimistic PV forecasts cause that the batteries tend to get filled more than they should, so it is common for them to overflow and, consequently, to result in penalties during these central hours. In this figure, it can also be seen in the top row that 10 days out of 15 presented less deviations and worked better with the stochastic optimization. The days whose deviations 80 Chapter 5. Chance Constraint Optimization Figure 5.5 Heatmap with the deviations from the contract in the deterministic approach. were reduced presented a total improvement of 35.45 kWh, with 24.68% of improvement on average for each day. On the contrary, the days which increased their deviations performed 12.88 kWh worse with 3.96% of increase on average for each day. The simulations executed show that the VPP is less operable when the battery charges are near their limits. According to figure 5.5, most of the deviations happened around the central hours since the storage virtualization is not capable enough absorbing the impact of the uncertainties due to the fact that the storage cannot work as a buffer. Thus, minimal deviations in the DML predictions may impact on the optimization at the control stage if the forecasting errors are persistent during peak hours. However, it can also be observed that the stochastic optimization improves the overall performance of the optimization for the majority of the cases. 5.2.3 Disturbance Mitigation Layer Robustness The accuracy of forecast systems improves when several sources contribute to the prediction. In fact, the higher the number of sources is, the better the worst case works. In practice, individual methods may have a significantly worse performance than any multi-source method in the worst case, and a slightly similar performance in the best one [147]. The disturbance predictor in the DML defined in this paper has two different sources, but the addition of other sources does not present any difficulty. The combination of different forecasts will provide the aggregation DML with independence, which implies a more robust system by limiting the impact of the disturbance predictors in the optimization problem. It can be seen in section 5.2.2 how the single activation of the ED component of the DML would provide better results than also activating the CC one, for the given case study. 5.3 Conclusions 81 However, this makes the optimization more dependent on ED errors, resulting in a less robust system for the majority of the executions. 5.3 Conclusions One of the most important problems regarding the use of renewables is the presence of uncertainties. The lack of accuracy in the forecast services can lead to instabilities in the system or to sanctions for not fulfilling the commitment signed at the time of bidding, before starting the operation stage. In many scenarios, the forecast sources are external, which cannot receive feedback from the EMS to improve their accuracy. Therefore, the operation of VPPs must be improved by modelling the potential disturbances and optimizing the bids to compensate them. A stochastic DML implementation on a MPC problem to mitigate the deviations is proposed. The resulting SMPC model improves the overall revenue of the grid by reducing the penalties at operation time. This implies that, although the optimization earnings could be lower than the ones when not applying the mitigation layer, the results show that the reduction of penalties will compensate these lower revenues. The results show that the implementation of the DML reduces the average error between the optimized profile and the operated one 11.85% with respect to the deterministic approach in the case study presented. In fact, the average error on injection (the one which implies the highest penalties) was also reduced in 11.1%, which generates large savings. It can also be obtained from the results that partial DML activation can be profitable as well, since the CC component improves the results in 9.67%, whereas the NN component improves them in 11.92%. These findings open new lines of research such as the implementation of a more intelligent CC component, or the development of even more accurate NNs for building more intelligent disturbance predictors. The approach of combining both components in the DML as well as the analysis of adding several more components to build a more robust stochastic agent are interesting subjects of study. 88 Chapter 6. The Pilot. Analysis in a real environment Table 6.1 Weather status during main hours for photovoltaic generation for August 25th, 2018. Source [148]. Time Temperature Humidity Condition 03:00 PM 16.33 C 59% Cloudy 04:00 PM 15.7 C 67% Cloudy 05:00 PM 14.3 C 63% Cloudy 06:00 PM 14.97 C 59% Cloudy 07:00 PM 13.32 C 72% Rain 08:00 PM 12.65 C 62% Cloudy 09:00 PM 12.3 C 62% Cloudy 10:00 PM 11.99 C 58% Cloudy 11:00 PM 11.32 C 62% Cloudy Figure 6.6 PA factor for every day during the 2-month sample time. To fully understand the figure 6.6 it is important to highlight that due to some problems, as well as the tunning of different parameters in the forecast algorithms, the forecasts were pessimistic about generation during the first days of July. This implied that the real injection was much higher than the forecasted profile defined in DA. Due to this, the grid stored more energy than needed and the batteries were filled up. This may not be a real problem, as the inverters can be configured not to inject when the batteries are full and there is still generation in the photovoltaic panel. In this case study, the decision was to inject all the excess energy, even if it meant not meeting the contract, due to the testing purposes of these scenarios, as well as to stress the sytem. 6.4 Robustness Assessment 89 Figure 6.7 Injection sample times prioritized over consumption sample times. Nevertheless, the platform worked very well even in such a hostile situation. August 2018 was a bad month for generation, which implied lack of accuracy in generation. In addition, it should be considered that the selected IP was Risky to get the best possible profit from the operation. It can be observed how, even with this aggressive IP, the last week of the 2-month time interval was much better than the previous ones, since the intelligent services and the infrastructure were improved with new developments. Another important key question that can be extracted from the pilot is that the strategy defined in the subsection 4.2.3, about the priority of meeting when injecting power, actually worked and reduced penalties. In figure 6.7 it can be observed how the execution of August 25th carried out two clear consume-to-inject actions, represented with dashed areas in the figure. The red dashed areas represent the sample times when the grid consumed more than the commitment to meet, as much as possible, the contract in sample times for power injection. It can also be observed how the injection target of the first green dashed area was achieved during the first following two hours. The main reasons for failure after those two hours are both the errors in the forecast of user consumption and deviations in the short-term forecast algorithm for generation. In the second consume-to-inject action, the bottlenecks are physical constraints. It is physically impossible to charge the batteries faster. Finally, at 20:00 PM the generation forecast was not accurate enough, so the system tried to meet the commitment without success. Nevertheless, a penalty minimization was achieved. 6.4 Robustness Assessment During this pilot, the system was stressed to assess how it would work in real environments where there were poor connections and many network errors. Several hardware failures 90 Chapter 6. The Pilot. Analysis in a real environment Figure 6.8 Average errors per hour / PA absolute deviation.. were also solved. The criteria for including them in the development of the final product were based on different priorities according to the severity of the issue, and its frequency of ocurrence. It can be observed how the number of connection errors is independent of the absolute deviation of PA. In fact, the absolute deviation of PA decreases with time. In addition, not all errors were due to real problems in the grid because some of them were explicitly forced to test the system under limiting and stressful conditions. It can be observed that the system works better in the last days, even with a higher average of errors per hour. Most of these errors are temporary, such as some delays or network problems. The main feature of the developments in this context was the ability of the inverter to store the 18 following setpoints from the last execution. This enabled the system to work temporarily offline (see subsection 3.1.3). 7 Framework as a platform Why is more important than how. Second Law of Software Architecture So ftware Engineering can be defined as the systematic approach to the development, operation, maintenance and retirement of software in accordance with IEEE[ 150 ]. Any valuable software development must meet different standards without losing the focus on the end user [ 151 , 152 , 153 ]. This chapter details design decisions, the software architecture, tools and models that have been built to serve the algorithms as a service. The platform has been implemented in python with Django Framework and the python scientific stack (Numpy, Scipy, Pandas, Scikit among others), and it is served through a secure REpresentational State Transfer (REST) API. It is prepared to run asynchronously all the algorithms, making the platform more robust and resilient against network delays or computational issues. The software which has been chosen to solve the algorithms is CPLEX, using its Python API available for the three main operating systems. The main reason why CPLEX was chosen is its natural integration with MATLAB (used to prototype the algorithms) so that both implementations can provide the same approximations when solving the equations. Developing in MATLAB is faster than developing in Python when working with mathematical problems, and this way of proceeding makes the development experience better and more reliable. The other powerful reason was the fact that CPLEX works very well with sparse matrices. This type of matrices only stores indices and values of non-zero cells. Due to the shape of the matrices in these optimization problems, there is a huge difference in both space and processing time between this type and the classical one (dense matrices) since any cell that does not belong to the constraint specification will be empty. As an example, the coefficient matrix size considering a problem with only three nodes for the first versions of Aggregated Day-Ahead (AGG-DA) is larger than 4GB. This slows down the process of working in workstations with less than 8GB of RAM and it is absolutely non-scalable. However, the sparse version was a bit larger than 100MB. A good implementation based on a good 91 92 Chapter 7. Framework as a platform design and supported by powerful tools are the key elements to make the implementation of powerful services possible. The platform is not agnostic regarding this decision, so changing the solver may involve some code refactor. Nevertheless, the architectural pattern organizes these potential changes in the last layer, reducing the impact when changing the solver. 7.1 Design Patterns and Software Architecture It is essential to treat the development process like any classical engineering discipline to build robust and solid platforms, which implies repeatability, rigor and effective analysis [ 154 ]. Software Engineering is not only coding, but it also involves working with high levels of abstraction, designing solutions considering end users and business engagement as well as dealing with the latest concepts of continuous delivery and continuous integration. Delivering quality software is impossible without defining the software architecture, technology stack integrations, or even the test plan to find future failures. In software design literature there are many universally accepted design patterns. A design pattern can be vaguely defined as a bundle of design decisions [ 155 ]. More precisely, they are often a set of standard solutions for various well-known scenarios. There are different fields of application for design patterns such as object-oriented programming [ 156 ], agile methodology [ 157 ], or even for complete architectures (Big Ball of Mud, Unitary Architecture or Client-Server Architecture) [ 154 , Part II]. The irruption of Internet of Things (IoT) has exploited the deployment and aceptation of decentralized architectures, where each device may have enough power to perform Extract, Load and Transform actions, relaxing the computational requirements of the central server in distributed environments. One of the main classifications regarding architecture is related to deployment: monolithic (systems that have been designed to be run in a single deployment unit, most of the times for unique users) and distributed (many units and users connected simultaneously by remote protocols). The second approach faces new challenges that were not present in monolitic designs, such as network reliability, network administration permission issues, the need of more secure firewalls, protocols and authentication systems or even new business models to build profitable solutions. The concept of cloud computing involves distributed architectures for scalable platforms. The most widespread one is the Client-Server architecture, also called Two Tier. The simplest scenario is composed of a server node and different nodes working as clients, such as browsers, mobile apps or IoT devices. The possibilities in this architecture are not limited, since the participant nodes can act as server and client simultaneously, and dynamically change their role depending on the use case. Nevertheless, complex environments do not need complex architectures most of the times, and the best option is usually to keep it simple and straightforward (KISS principle). Figure 7.1 shows several of the main actors in cloud environments. There are many more, but these concepts have been selected due to their importance and the relevancy in relation to the developed platform. The figure shows how the requests find the entrypoint through the Load Balancer. This node, which can also be multiple or belong to a hierarchy [ 158 ] automatically distributes the incoming traffic across different instances depending 7.2 Platform Definition 93 Figure 7.1 Example of a Cloud Computing Software as a Service (SaaS) with elastic servers with different servers for business logic, databases and the concept of load balancer. on a defined balancing criteria. These instances are created to parallelize server requests and to send the response faster. Despite their server nature in this sample architecture, the balancers can also act as individual clients regarding other servers where other services may require specific licences or stronger security restrictions. They can also be in the need of being considered as critical legacy applications which must be isolated from the new layers. The flexibility of the cloud allows communication between platforms from different decades and requirements for systems that have compatible internet protocols. In this example, the four central instances improve the system scalability for a large number of requests per second. Elastic cloud computing enhances resource parallelization and dynamic infrastructure scaling, improving investment and maintenance costs for relatively large platforms [ 159 ]. There are two main possibilities for scaling an infrastructure: vertical scaling (by adding more power to the infrastructure, such as RAM or CPU, to an existing machine) or horizontal scaling (by adding more machines into the pool of resources). The scalability could be done manually if the traffic peak pattern were correctly identified and the volume of traffic were not very large. However, most of the times an algorithm will be needed to handle the scale changes optimally. The latest approaches in the literature are the use of smart agents developed with reinforcement learning [160]. 7.2 Platform Definition Amongst the different architectures present in the literature [ 154 , Part II], it has been chosen a mixed Service-Based Architecture, which is distributed, combined with a strong sense of layer isolation in the implementation of each service. It could be considered that each service has been designed by using the Layered Architecture, which is monolithic. The reason why this hybrid architecture has been chosen rather than others is to combine the simplicity of layered implementation with the flexibility of the service-base. The former improves scalability if necessary, and the smart agent can be deployed in different 94 Chapter 7. Framework as a platform instances (depending on the service, as it is shown in the figure 7.2). The latter reduces possible deployment issues and maintenance costs, since bugs and enhancements will be easier to close. The following subsections explain the contributions of every architecture to integrate the hybrid version. 7.2.1 Layered Architecture Component Figure 7.2 shows in its first column a theoretical approach of layered architecture, differentiating the presentation and database layers from the application business logic. Persistence layers are often abstractions to automate database query building. Figure 7.2 Application of Layered Architecture to framework development. In the second column, it can be observed the equivalence between theory and actual implementation whereas the use of the different technologies in the different layers is shown in the third column. The services are accessible through a secure API REST and they can be consumed from front-end applications, such as native mobile apps, progressive web applications, other servers or IoT devices able to make REST requests. The API is built using Django and Django Rest Framework (DRF). Django is a high-level Python web framework for building platforms with a clean and pragmatic code design. The main feature of Django is its Object-Relational Mapping (ORM), placed in the persistence layer. The ORM is the interface between the database and Python objects and vice versa. DRF is a library built on Django which exposes the endpoints and the services through the API with its APIViews by using JavaScript Object Notation (JSON). The JSONs are deserialized using the DRF serializers and built into Python objects. Figure 7.3 shows in detail the entire flow of an algorithm call from a client. Asynchronous execution is explained in the section 7.4. This separation is necessary to avoid the dangerous concept of coupling, defined as the degree of interdependence between software modules. The flow of a sample request is as follows: 1. The system receives the request to the URL exposed through the ApiView. 7.2 Platform Definition 95 Figure 7.3 Algorithm execution flow over the Layered Architecture. 2. The ApiView creates the serializer and validates it. After validating the data according to the API definition, the serializer creates Python objects from the received JSON. 3. The ApiView goes one level deeper and calls the main service once the data is transformed. 4. The main services can call other auxiliary services at the same level or work with the models and the persistence layer. The system send the data to a queue to solve the algorithm asynchronously once the mathematical problem is fully built to be solved. 5. The algorithm layer solves the instance of the problem during its turn in the queue, and stores the solution in the database. 7.2.2 Service-Based Architecture Component Service-based architectures are generally distributed architectures where the main important concept for designing, implementing and performing the functionality of the platform is the service itself. Components tend to be more isolated, leading to more decoupled, testable and easier-to-maintain software. Figure 7.4 shows a sample service-based architecture composed of different nodes with independently deployed services, but working with the same information source. Achieving this ideal scenario is not trivial since there are usually strong dependencies between the services. In these cases, design patterns help to decrease the decoupling, but sometimes there would be inevitable relations. There is no universal solution in the dilema of duplicating the code to improve the isolation or maintaining the dependency; thus, a contextualization becomes necessary for analysts to be able to choose the best option. The framework has been designed to have isolated but synchronized services. It has also been explained that even several executions of the same algorithm are also completely independent from each others. 96 Chapter 7. Framework as a platform According to [ 154 , Chapter 13], in most cases there is not more than one instance per service unless some scalability or fault tolerance issues are detected. Another important aspect mentioned in this reference is the small number of services that usually share a context: between 4 and 12, with an average of around 7. There are currently 4 different services that can be deployed in different nodes: • Management Service: Unit to maintain settings and grid information. This service is used to perform creating, reading, updating and deleting operations on the grids, energy nodes, storages and users stored in the platform. These operations are commonly known as CRUD operations. • Day Ahead Service: This service unit contains the four implemented versions of DA algorithms. Simulation, submission and parameter tweaking for this service. It also manages the DIP at optimization time. • Penalty Reduction Service: Contains the functionalities related to PR algorithm and the activation / deactivation of the extra-optimizations. • Demand Response Service: This service performs the optimizations related to the participation in the DRP. Figure 7.4 Application of Service-Based Architecture to framework development. 7.3 Software Model Design The proposed model is a conceptual design to show the big picture. This design was thought to get the benefits of relational databases and non relational databases, depending on the nature of the stored data. Relational databases are digital databases which are organized by tables which can present relations to other tables and constraints regarding either data or relations. They are very well structured. In addition, the information must meet the definition in terms of number of fields, types and validations. The data is managed by using Structure Query Language (SQL) queries, but the syntax can be slightly different depending on the database 7.3 Software Model Design 97 distribution (for example: Oracle DB, MySQL DB or Microsoft SQL). They can be used to manage transaction-oriented applications and they support ACID transactions (Atomicity, Consistency, Isolation, Durability). Non-relational databases, also called NoSQL, emerged in early 2000s, and they are usually grouped into four categories: key-value stores, graph stores, column stores and document stores. Some of the most popular ones are MongoDB, DocumentDB Cassandra or Redis. They work with semi-structured data (XML, JSON) and they are prepared to handle large amount of data. Their use has enormously increased since the irruption of data analysis and artificial intelligence in the industry. NoSQL databases are mainly used to store information with many records that are supposed to be stored only once and read multiple times. The information can be updated or deleted but it is not supposed to change in the future. On the other hand, relational databases are designed for storing smaller databases with more volatile information. However, a table can handle several million rows, depending on the distribution. Figure 7.5 describes the conceptual proposal of the platform model. All the entities are presented in a relational representation. The implementation may differ from design due to technological limitations, but the key point in this struture is the use of a hybrid database system to store and manage the information. The items stored in the relational database have an orange background color. They are mainly configuration items for the grid, energy nodes etc. Django ORM is transparent when choosing the SQL database. The one selected for the platform was Maria DB, an open source relational database very similar to MySQL, but with the enhancements from community contributions. The elements located in NoSQL databases have a grey background color. They are mainly measurements and algorithms executions. MongoDB is the selected NoSQL database engine, a document-based database integrable with Django that fits the needs of this architecture. The use of NoSql database is necessary due to the amount of data that will be stored. In the data age, storing information is very important for smart agents and big data analysis since it adds some extra value to the platform. For example, a full day performance for 40 nodes (considering only the 144 steps of the PR algorithm) will create 103,680 records for short time forecast generation, and the same amount for load forecast. This would make a SQL database inoperable in a few months, or even less, if larger grids, or different simultaneous grids, were operated. List of Figures 1.1 Example of a classic electrical system 2 1.2 Schematized figure of VPP operation framework. Source: Slightly modified version from [12] 4 1.3 Stakeholders involved in the electricity system 5 1.4 Market with retail competition (Source [14]) 6 2.1 Progress towards renewable energy source targets at Member State and EU-28 levels (source [23]) 14 2.2 DER technologies (source [27]) 15 2.3 Methods of energy storage according to form of energy (source [40]) 18 2.4 DAM process 19 2.5 DA market and real time balancing integration steps 19 2.6 Full picture of DA market and real time balancing integration 19 2.7 DRPs classification 21 2.8 Different uncertainty sources 24 2.9 Perception-Action Cycle 24 2.10 Simple Perceptron 27 2.11 Multilayer Perceptron 28 2.12 Sample of a deep CNN presented in [103] with an extremely good performance on image classification 28 2.13 Graphical scheme of the transformation process of an image in a CNN 29 2.14 Standard NN (left) and NN after applying dropout (right), source [104] 29 2.15 Noise reductor autoencoder sample with a sample of a number from MNIST database [106] 30 2.16 Schematized figure of a simple recurrent neural network 30 2.17 Long Short Term Memory Neural Network cell scheme 31 2.18 Gated Recurrent Unit cell scheme 32 2.19 Generative adversarial network scheme 32 2.20 Realistic artificial faces built by a generative adversarial network 33 105 106 List of Figures 3.1 Energy Node definition and integration in the grid 36 3.2 Piped Baseline with a DR call at 8 am. 39 3.3 Conservative IP (left) and Risky IP (right) 41 3.4 Dynamic Battery Virtualization with Mixed IP 41 3.5 Platform General Overview 43 3.6 2-stage hierarchical framework execution example 43 4.1 Example of an optimal grid profile (baseline) for a day with good generation and a low energy-demanding VPP 46 4.2 SO types. On the left, an energy producer SO; on the right, a market participant-producer SO 47 4.3 Profile optimization without P2P 51 4.4 Profile optimization with P2P 52 4.5 Intent profile comparative 53 4.6 Full day simulation for Risky IPs without any generation 54 4.7 Full day simulation for Conservative IPs without any generation 54 4.8 Full day simulation for Mixed IPs without any generation 55 4.9 SOC evolution for a Risky IP execution 56 4.10 SOC evolution for a Conservative IP execution 56 4.11 SOC evolution for a Mixed IP execution 57 4.12 Comparison between power mean and real setpoints 60 4.13 Complete day simulation without generation in any node 62 4.14 Execution with Intrahour Equilibrium Off 63 4.15 Execution with Intrahour Equilibrium On 64 5.1 Encoder-Decoder Architecture 75 5.2 Pessimistic generation forecasting (red) vs Optimistic generation forecasting (green) vs Real generation values 77 5.3 Sample of disturbance forecasting with CC and ED 78 5.4 Heatmap with the cardinal differences between the MSE with deterministic and stochastic versions of the problem. The top horizontal bar shows the average improvement for the whole day 79 5.5 Heatmap with the deviations from the contract in the deterministic approach 80 6.1 Aggregated Power Injection profile (green) vs Aggregated Power Consumption profile per day (red) 84 6.2 Real DA optimization for a single day (August 25th, 2018) 85 6.3 Forecast of energy price per hour to participate in DA 85 6.4 Number of complete battery cycles for each participant node 86 6.5 DA optimization vs real execution 87 6.6 PA factor for every day during the 2-month sample time 88 6.7 Injection sample times prioritized over consumption sample times 89 6.8 Average errors per hour / PA absolute deviation. 90 List of Figures 107 7.1 Example of a Cloud Computing SaaS with elastic servers with different servers for business logic, databases and the concept of load balancer 93 7.2 Application of Layered Architecture to framework development 94 7.3 Algorithm execution flow over the Layered Architecture 95 7.4 Application of Service-Based Architecture to framework development 96 7.5 Application proposed model detailing SQL entities (orange) and NoSQL entities (gray) 98 List of Tables 1.1 Increases in EU final energy consumption from 2014 to 2016 in 3 2.1 Brief historical timelines of the development of photovoltaic technology (source: [32]) 16 2.2 Comparison between the efficiencies of renewable generation sources (source: [34]) 16 3.1 Nodes Definition 37 4.1 Relative deviation compared with best execution 55 5.1 Encoder-Decoder Hyperparameters 76 5.2 Different executions with the complete/partial activation, or deactivation, of the DML 79 6.1 Weather status during main hours for photovoltaic generation for August 25th, 2018. Source [148] 88 109 Listings 5.1 Implementation of CC model 74 111 Bibliography [1] Electrical Engineering Portal. Standard line voltages. https:// electrical-engineering- portal.com/ current-systems-voltage-levels. Accessed: 2022-05-21. [2] Shoemaker T.M. and Mack J.E. Lineman’s and Cableman’s Handbook, Thirteenth Edition. McGraw-Hill Education, New York, 2017. [3] Wu Y., Chang S.M., and Hu Y. Literature review of power system blackouts. Energy Procedia, 141:428 – 431, 2017. Power and Energy Systems Engineering. [4] Thomas S. and Rosenow J. Drivers of increasing energy consumption in europe and policy implications. Energy Policy, 137:111108, 2020. [5] McLellan B., Florin N., Giurco D., Kishita Y., Itaoka K., and Tezuka T. Decentralised energy futures: The changing emissions reduction landscape. Procedia CIRP, 29:138 – 143, 2015. The 22nd CIRP Conference on Life Cycle Engineering. [6] Vural G. Renewable and non-renewable energy-growth nexus: A panel data application for the selected sub-saharan african countries. Resources Policy, 65:101568, 2020. [7] McGee J.A. and Greiner P.T. Renewable energy injustice: The socio-environmental implications of renewable energy consumption. Energy Research and Social Science, 56:101214, 2019. [8] Thombs R.P. Has the relationship between non-fossil fuel energy sources and co2 emissions changed over time? a cross-national study, 2000–2013. Climate Change, 56:101214, 2019. [9] European Renewable Energy Council. Mapping renewable energy pathways towards 2020. http:// www.eufores.org/ fileadmin/ eufores/ Projects/ REPAP_2020/ EREC- roadmap-V4.pdf . Accessed: 2022-05-21. 113 120 Bibliography [87] Almeida R.H., Carrêlo I.B., Lorenzo E., Narvarte L., Fernández-Ramos J., Martínez- Moreno F., and Carrasco L.M. Development and test of solutions to enlarge the power of pv irrigation and application to a 140 kw pv-diesel representative case. Energies, 11(12):3538, 2018. [88] Ghiani E., Pilo F., and Celli G. Chapter 1 - definition of smart distribution networks. In Operation of Distributed Energy Resources in Smart Distribution Networks, pages 1 – 23. Academic Press, 2018. [89] Vergara-Dietrich J.D., Morato M.M., Mendes P.R.C., Cani A.A., Normey-Rico J.E., and Bordons C. Advanced chance-constrained predictive control for the efficient energy management of renewable power systems. Journal of Process Control, 74:120 – 132, 2019. Efficient energy management. [90] Russell S. and Norvig P. Artificial Intelligence: A Modern Approach. Prentice Hall Press, USA, 3rd edition, 2009. [91] Turing A.M. I.—COMPUTING MACHINERY AND INTELLIGENCE. Mind, LIX(236):433–460, 10 1950. [92] Wilson R. and Keil F. The MIT Encyclopedia of the Cognitive Sciences, volume CXXXII. Cambridge, Mass. : MIT Press, 01 1999. [93] Cutsuridis V. Cognitive models of the perception-action cycle: A view from the brain. In The 2013 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2013. [94] Bayes and Price. An essay towards solving a problem in the doctrine of chances. by the late rev. mr. bayes, f. r. s. communicated by mr. price, in a letter to john canton, a. m. f. r. s. Philosophical Transactions (1683-1775), 53:370–418, 1763. [95] Nield T. Is another ai winter coming? https:// medium.com/ hackernoon/ is-another- ai-winter-coming-ac552669e58c, feb 2019. Accessed: 2022-05-21. [96] Mahanta J. Introduction to neural networks, advantages and applications. https://towardsdatascience.com/ introduction-to-neural-networks-advantages- and-applications-96851bd1a207, jul 2017. Accessed: 2022-05-21. [97] Cybenko G. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals, and Systems (MCSS), 2(4):303–314, December 1989. [98] Leshno M., Lin V.Y., Pinkus A., and Schocken S. Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Networks, 6(6):861–867, 1993. [99] Pinkus A. Approximation theory of the mlp model in neural networks. ACTA NUMERICA, 8:143–195, 1999. [100] Zhou D. Universality of deep convolutional neural networks. Applied and Computational Harmonic Analysis, 48, 06 2019. Bibliography 121 [101] Schäfer A.M. and Zimmermann H.G. Recurrent neural networks are universal approximators. In Proceedings of the 16th International Conference on Artificial Neural Networks - Volume Part I, ICANN’06, page 632–640, Berlin, Heidelberg, 2006. Springer-Verlag. [102] Lu Z., Pu H., Wang F., Hu Z., and Wang L. The expressive power of neural networks: A view from the width. In Advances in Neural Information Processing Systems 30, pages 6231–6239. Curran Associates, Inc., 2017. [103] Krizhevsky A., Sutskever I., and Hinton G.E. Imagenet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, NIPS’12, page 1097–1105, Red Hook, NY, USA, 2012. Curran Associates Inc. [104] Srivastava N., Hinton G., Krizhevsky A., Sutskever I., and Salakhutdinov R. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15:1929–1958, 06 2014. [105] Stanford University. Autoencoders. http:// ufldl.stanford.edu/ tutorial/ unsupervised/ Autoencoders/ . Accessed: 2022-05-22. [106] LeCun Y. and Cortes C. MNIST handwritten digit database. http:// yann.lecun. com/ exdb/ mnist/ , 2010. Accessed: 2022-05-22. [107] Bilski J., Rutkowski L., Smoląg J., and Tao D. A novel method for speed training acceleration of recurrent neural networks. Information Sciences, 2020. [108] Hochreiter S. and Schmidhuber J. Long short-term memory. Neural computation, 9:1735–80, 12 1997. [109] Cho K., van Merriënboer B., Gulcehre C., Bahdanau D., Bougares F., Schwenk H., and Bengio Y. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724–1734, Doha, Qatar, October 2014. Association for Computational Linguistics. [110] Goodfellow I., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair S., Courville A., and Y. Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems 27, pages 2672–2680. Curran Associates, Inc., 2014. [111] Salimans T., Goodfellow I., Zaremba W., Cheung V., Radford A., Chen X., and Chen X. Improved techniques for training gans. In Advances in Neural Information Processing Systems 29, pages 2234–2242. Curran Associates, Inc., 2016. [112] Isola P., Zhu J., Zhou T., and Efros A.A. Image-to-image translation with conditional adversarial networks, 2018. [113] Radford A., Metz L., and Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks, 2016. 122 Bibliography [114] Jin Y., Zhang J., Li M., Tian Y., Zhu H., and Fang Z. Towards the automatic anime characters creation with generative adversarial networks, 2017. [115] Karras T., Aila T., Laine S., and Lehtinen J. Progressive growing of gans for improved quality, stability, and variation, 2018. [116] Okur Ö., Voulis N., Heijnen P., and Lukszo Z. Aggregator-mediated demand response: Minimizing imbalances caused by uncertainty of solar generation. Applied Energy, 247:426–437, 2019. [117] Bemporad A. and Morari M. Control of systems integrating logic, dynamics, and constraints. Automatica, 35(3):407 – 427, 1999. [118] Burger S., Chaves Ávila J.P., Batlle C., and Pérez-Arriaga I.J. A review of the value of aggregators in electricity systems. Renewable and Sustainable Energy Reviews, 77:395 – 405, 2017. [119] del Real A.J., Arce A., and Bordons C. An integrated framework for distributed model predictive control of large-scale power networks. IEEE Transactions on Industrial Informatics, 10(1):197–209, 2014. [120] Garcia-Torres F., Valverde L., and Bordons C. Optimal load sharing of hydrogenbased microgrids with hybrid storage using model-predictive control. IEEE Transactions on Industrial Electronics, 63(8):4919–4928, 2016. [121] Amprion. Ausgleichsenergieabrechnung gegenber bilanzkreisverantwortlichen. https:// www.amprion.net/ Strommarkt/ Bilanzkreise/ Ausgleichsenergiepreis/ . Accessed: 2022-05-21. [122] Qian L.P., Zhang Y.J.A., Huang J., and Wu Y. Demand response management via real-time electricity price control in smart grids. IEEE Journal on Selected Areas in Communications, 31(7):1268–1280, 2013. [123] van Ackooij W. A discussion of probability functions and constraints from a variational perspective. Set-Valued and Variational Analysis, 28(4):585–609, 2020. [124] González T., Henrion R., and Perez-Aros P. Dynamic probabilistic constraints under continuous random distributions. Mathematical Programming, pages 1–32, 2020. [125] Farina M. and Scattolini R. Model predictive control of linear systems with multiplicative unbounded uncertainty and chance constraints. Automatica, 70:258 – 265, 2016. [126] Ke H. and Liu B. Project scheduling problem with mixed uncertainty of randomness and fuzziness. European Journal of Operational Research, 183(1):135 – 147, 2007. [127] Zhou J., Yang F., and Wang K. Multi-objective optimization in uncertain random environments. Fuzzy Optimization and Decision Making, 13(4):397–413, 2014. Bibliography 123 [128] Chen J., Sun Q., and Shi Y. Stochastic self-triggered mpc for linear constrained systems under additive uncertainty and chance constraints. Information Sciences, 459:198 – 210, 2018. [129] Pantula P.D. and Mitra K. A data-driven approach towards finding closer estimates of optimal solutions under uncertainty for an energy efficient steel casting process. Energy, 189:116253, 2019. [130] Zhou Y., Yu W., Zhu S., Yang B., and He J. Distributionally robust chanceconstrained energy management of an integrated retailer in the multi-energy market. Applied Energy, 286:116516, 2021. [131] Ugranlı F. Probabilistic distribution planning: Including the interactions between chance constraints and renewable energy. Sustainable Energy, Grids and Networks, 23:100372, 2020. [132] Wu C.B., Huang G.H., Li W., Xie Y.L., and Xu Y. Multistage stochastic inexact chance-constraint programming for an integrated biomass-municipal solid waste power supply management under uncertainty. Renewable and Sustainable Energy Reviews, 41:1244 – 1254, 2015. [133] Ning C. and You F. Optimization under uncertainty in the era of big data and deep learning: When machine learning meets mathematical programming. Computers & Chemical Engineering, 125:434 – 448, 2019. [134] Marino C.A. and Marufuzzaman M. A microgrid energy management system based on chance-constrained stochastic optimization and big data analytics. Computers & Industrial Engineering, 143:106392, 2020. [135] Moosavi A., Rao V., and Sandu A. Machine learning based algorithms for uncertainty quantification in numerical weather prediction models. Journal of Computational Science, 50:101295, 2021. [136] Yin J., Medellín-Azuara J., Escriva-Bou A., and Liu Z. Bayesian machine learning ensemble approach to quantify model uncertainty in predicting groundwater storage change. Science of The Total Environment, 769:144715, 2021. [137] Guevara E., Babonneau F., Homem de Mello T., and Moret S. A machine learning and distributionally robust optimization framework for strategic energy planning under uncertainty. Applied Energy, 271:115005, 2020. [138] Westermann P. and Evins R. Using bayesian deep learning approaches for uncertainty-aware building energy surrogate models. Energy and AI, 3:100039, 2021. [139] Shen G., Tan Q., Zhang H., Zeng P., and Xu J. Deep learning with gated recurrent unit networks for financial sequence predictions. Procedia Computer Science, 131:895 – 903, 2018. Recent Advancement in Information and Communication Technology:. 124 Bibliography [140] Loshchilov I. and Hutter F. Decoupled weight decay regularization, 2017. [141] Morales J., Conejo A., Madsen H., Pinson P., and Zugno M. Integrating Renewables in Electricity Markets - Operational Problems. Springer US, 01 2014. [142] Moreno M.A., Bueno M., and Usaola J. Evaluating risk-constrained bidding strategies in adjustment spot markets for wind power producers. International Journal of Electrical Power & Energy Systems, 43(1):703 – 711, 2012. [143] Réseau de Transport d’Électricité. Imbalance settlement price calculation. https:// www.services-rte.com/ en/ learn-more-about-our-services/ becoming- a-balance-responsible-party/ Imbalance-settlement-price.html, 2022. Accessed: 2022-05-22. [144] Elia Group. Imbalance prices (15 min). https:// www.elia.be/ en/ grid-data/ balancing/ imbalance-prices-15-min, 2022. Accessed: 2022-05-21. [145] Nord Pool. Regulating prices. https:// www.nordpoolgroup.com/ en/ Market-data1/ #/ nordic/ table, 2022. Accessed: 2022-05-21. [146] UK Power Networks. Photovoltaic (pv) solar panel energy generation data. https:// data.london.gov.uk/ dataset/ photovoltaic--pv--solar-panel-energy- generation-data, 2017. Accessed: 2022-05-21. [147] Hibon M. and Evgeniou T. To combine or not to combine: selecting among forecasts and their combinations. International Journal of Forecasting, 21(1):15 – 24, 2005. [148] Open Weather Map. Historical weather data in borkum (germany). https: // openweathermap.org/ . Accessed: 2022-05-21. [149] Wunderground. Historical weather data in borkum (germany). https://www. wunderground.com. Accessed: 2022-05-21. [150] IEEE. IEEE standard glossary of software engineering terminology, 1990. [151] Chammas A., Quaresma M., and Mont’Alvão C. A closer look on the user centred design. Procedia Manufacturing, 3:5397 – 5404, 2015. 6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015) and the Affiliated Conferences, AHFE 2015. [152] Dong Y., Zhu R., Tian Q., Liu W., and Peng W. A scenario interaction-centered conceptual information model for ux design of user-oriented product-service system. Procedia CIRP, 83:335 – 338, 2019. 11th CIRP Conference on Industrial Product- Service Systems. [153] Bullinger H., Bauer W., Wenzel G., and Blach R. Towards user centred design (ucd) in architecture based on immersive virtual environments. Computers in Industry, 61(4):372 – 379, 2010. Human-Centered Computing Systems in Industry - A Special Issue in Honor of Professor G. Salvendy. Bibliography 125 [154] Richards M. and Ford N. Fundamentals of Software Architecture. O’Reilly Media, Inc., 2020. [155] Bass L., Clements P., and Kazman R. Software Architecture in Practice. Addison- Wesley Professional, 3rd edition, 2012. [156] Gamma E., Helm R., Johnson R., and Vlissides J. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley Longman Publishing Co., Inc., USA, 1995. [157] Martin R.C. Agile Software Development: Principles, Patterns, and Practices. Prentice Hall PTR, USA, 2003. [158] Afzal S. and Ganesh K. Load balancing in cloud computing -a hierarchical taxonomical classification. Journal of Cloud Computing, 8, 12 2019. [159] Caballer M., Alfonso C., Alvarruiz F., and Moltó G. Ec3: Elastic cloud computing cluster. Journal of Computer and System Sciences, 78:1341–1351, 12 2013. [160] Rossi F., Nardelli M., and Cardellini V. Horizontal and vertical scaling of containerbased applications using reinforcement learning. In 2019 IEEE 12th International Conference on Cloud Computing (CLOUD), pages 329–338, 2019. [161] Barber S. How fast does a website need to be? http:// www.perftestplus.com/ resources/ how_fast.pdf , 2010. Accessed: 2022-05-321. [162] Nielsen J. Usability Engineering. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1993. [163] Celery official webpage. https:// docs.celeryq.dev/ en/ stable/ index.html. Accessed: 2022-05-21.