D4.1 Message-Based Communication Implemented

Grant Agreement: 101069510 Dissemination level: PU Page 1 of 61 D4.1 Message-Based Communication Implemented This work has been co-funded by the European Union’s Horizon Innovation Actions under grant agreement No. 101069510 Grant Agreement: 101069510 Dissemination level: PU Page 2 of 61 DOCUMENT INFORMATION WP number and title WP4 – EDDIE Interoperable Communication Layer Deliverable number D4.1 Version Number 1.3 Document Reference Doi: 10.5281/zenodo.11638318 Lead Beneficiary PON Deliverable type DEM – Demonstrator, pilot, prototype Planned deliverable date 30/06/2024 Date of Issue 29/06/2024 Dissemination level PU Author(s) FHO, PON Contributor(s) FHO, AIT, D4G, ENT, PON Keywords Interoperable Communication Layer, Region Connector, Message-based Communication Legal Disclaimer This work has been co-funded by the European Union’s Horizon Innovation Actions under grant agreement No. 101069510. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the granting authority European Climate, Infrastructure and Environment Executive Agency (CINEA). Neither the European Union nor the granting authority can be held responsible for them. The information in this document is provided “as is”, and no guarantee or warranty is given that the information is fit for any particular purpose. The below referenced consortium members shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials subject to any liability which is mandatory due to applicable law. © 2023-2025 by the EDDIE Consortium. Disclosure Statement The information contained in this document is the property of the EDDIE Consortium and it shall not be reproduced, disclosed, modified or communicated to any third parties without the prior written consent of the abovementioned entities. Grant Agreement: 101069510 Dissemination level: PU Page 3 of 61 CONSORTIUM PARTNERS The EDDIE Consortium consists of the following partners: Participant No Participant organisation name Short Name Country 1 University of Applied Sciences Upper Austria – Campus Hagenberg – Research and Development FHO AT 2 Copenhagen School of Energy Infrastructure, Department of Economics, Copenhagen Business School CBS DK 3 European University Institute EUI IT 4 University of Vienna, Faculty of Computer Science, Cooperative Systems Research Group VIE AT 5 Austrian Institute of Technology, Center for Digital Safety & Security, Competence Unit Cooperative Digital Technologies AIT AT 6 The Lisbon Council for Economic Competitiveness and Social Renewal asbl LIC BE 7 PONTON GmbH PON DE 8 Asociación de Empresas de Energía Eléctrica (aelec) AEL ES 9 DEDA – Public Gas Distribution Networks – Single Member S.A. DED GR 10 EDA Energiewirtschaftlicher Datenaustausch GmbH EDA AT 11 Südtiroler Energieverband SEV IT 12 FlexiDAO FLE ES 13 Digital4Grids D4G FR 14 EASEE Gas EAS FR 15 Entarc.eu ENT AT 16 ETA+ GmbH ETP DE Grant Agreement: 101069510 Dissemination level: PU Page 4 of 61 DOCUMENT HISTORY Version Date Status Author(s), Reviewer Description V0.1 20/02/2024 Draft Marc Kurz Document Template V0.2 27/02/2024 Draft Marc Kurz Draft Outline V0.3 08/03/2024 Draft Marc Kurz Table of Contents V0.4 20/03/2024 Draft Marc Kurz, Fabian Haas, Henner Carl Input collected V1.0 24/05/2024 Review Marc Kurz Draft sent to reviewers V1.1 03/06/2024 Review Stefan Kienler Document reviewed V1.2 14/06/2024 Pre-Final Marc Kurz Incorporated reviewer’s comments V1.3 18/06/2024 Final Marc Kurz Document finalized Grant Agreement: 101069510 Dissemination level: PU Page 5 of 61 AUTHOR AND REVIEWER ACKNOWLEDGMENTS Author names Partner Marc Kurz FHO Henner Carl PON Fabian Haas FHO Shievam Kashyap FHO Stefan Grünberger FHO Reviewer names Partner Georg Hartner ENT Stefan Kienler EDA Grant Agreement: 101069510 Dissemination level: PU Page 6 of 61 DEFINITIONS, ACRONYMS AND ABBREVIATIONS Acronyms/ Abbreviations Description AIIDA Administrative Interface for In-house Data Access API Application Programming Interface BPMN Business Process Model and Notation CAS Central Access Server CCM Customer Consent Management CCMO Customer Consent Management Online Process CIM Common Information Model CMS Central Market System DAP Data Access Provider DEP Data Exchange Platform DSO Distribution System Operator EDA Energiewirtschaftlicher Datenaustausch EDSN Energie Data Services Nederland EMIF Elhub Messaging Interface Grant Agreement: 101069510 Dissemination level: PU Page 7 of 61 EP Eligible Party HVD Historical Validated Data JSON JavaScript Object Notation MDA Metered Data Administrator MQTT Message Queuing Telemetry Transport OBIS Object Identification System PA Permission Administrator RC Region Connector REST Representational State Transfer SMGA Smart Meter Gateway Administrator TSO Transmission System Operator VHMD Validated Historical Metering Data XML Extensible Markup Language Grant Agreement: 101069510 Dissemination level: PU Page 8 of 61 EXECUTIVE SUMMARY Deliverable D4.1 of the EDDIE project, "Message-Based Communication Implemented," marks a significant milestone in developing an interoperable communication layer for European energy data exchange. This document, part of Work Package 4, addresses the varied methodologies across EU member states, ensuring interoperability, security, and reliability. The main objectives include developing a communication layer that bridges the gap between varying national implementations of EU-regulated processes, focusing on communication protocols and data formats. The aim is to create a uniform architecture for configuring the communication layer, enabling integration across different business processes and ensuring compliance with current and future standards. The initial architecture outlined in the grant agreement serves as the foundation for the communication layer. The focus is on creating a standardized, secure, and reliable infrastructure for data exchange, incorporating state-of-the-art analysis and legal framework considerations to ensure compliance and functionality across different regions. The core contribution of this deliverable is the message-based communication, with a particular focus on the Austrian case, utilizing the AS4 messaging protocol for data exchange. Detailed processes for customer consent management and data transmission are illustrated through BPMN diagrams. The region connector architecture serves as the gateway for data flow into the EDDIE Framework. Middleware systems like Ponton X/P messenger are used for AS4 communication in Austria, ensuring consistent and secure interactions despite regional differences in datasharing infrastructure. Technological integration involves the incorporation of different communication methods, including REST API and streaming-based communication, to cover a broad spectrum of European solutions. The EDDIE framework is designed for extensibility, supporting the addition of other messaging solutions. The D4.1 deliverable represents a critical step towards a unified, interoperable communication layer for European energy data exchange. By addressing diverse methodologies and aligning with stakeholder needs, the EDDIE project aims to enhance data exchange efficiency, security, and interoperability across the European energy landscape. Grant Agreement: 101069510 Dissemination level: PU Page 9 of 61 TABLE OF CONTENTS 1!INTRODUCTION .................................................................................................................. 13! 1.1!Purpose and Background of the Document .................................................................................... 13! 1.2!Scope and Intended Audience ................................................................................................................. 13! 1.3!Structure of the Document .......................................................................................................................... 14! 2!Communication Layer Design ..................................................................................... 15! 2.1!Introduction ............................................................................................................................................................ 15! 2.2!Initial Architecture .............................................................................................................................................. 15! 2.3!State of the Art ...................................................................................................................................................... 17! 2.3.1!Legal Framework ....................................................................................................................................... 17! 2.3.2!State of the art in the EU ....................................................................................................................... 19! 2.4!Architectural Design and Development Progress ...................................................................... 28! 2.5!General Architecture ....................................................................................................................................... 34! 2.5.1!Layered Architecture Concept ........................................................................................................ 35! 2.5.2!Connection of other Messaging Solutions ............................................................................. 35! 2.5.3!Region Connector Architecture ..................................................................................................... 35! 2.5.4!Technical Implementation ................................................................................................................ 36! 3!Message-Based Communication ............................................................................. 38! 3.1!Introduction ........................................................................................................................................................... 38! 3.2!Region Connector Austria ........................................................................................................................... 39! 3.2.1!Introduction .................................................................................................................................................. 39! 3.2.2!EDA Processes ............................................................................................................................................ 40! 3.2.3!Exchanged market documents .................................................................................................... 44! 3.2.4!Setup/Onboarding ................................................................................................................................. 49! 3.2.5!PONTON X/P Messenger ...................................................................................................................... 49! 3.3!Exemplary Message-based Request ................................................................................................... 51! 4!Outlook & Roadmap ....................................................................................................... 56! Grant Agreement: 101069510 Dissemination level: PU Page 16 of 61 To allow for an integration across business processes that are implemented differently per Member State, a uniform architecture is needed to configure the communication layer by process participants in such a way that interoperability, a high level of security and reliability is reached for use cases. At the same time, the data exchange layer needs to be aligned with current and future standards and best-practices regarding the underlying software frameworks and processes. The goal is a coherent, consistent, and configurable functional basis for communication protocols, configurable and usable primarily by “region connector” modules. The original EDDIE architectural schema, as outlined and detailed in the grant agreement, can be conceptualized through Figure 1. At its core lies the "EDDIE Interoperable Communication Layer," strategically positioned within the EDDIE Framework. This layer is included to address the wide array of European energy data exchange methodologies that the reference processes and national connectors are expected to utilize. Its primary objective is to ensure interoperability, essentially serving as a bridge between the diverse communication protocols and data formats employed by different European regions. The overarching aim of this communication layer is to cater to stakeholder requirements pertaining to the various types of data exchange necessary. The functionality of this layer is contingent upon the specific procedures required to access the relevant data families within the designated regions. This necessitates the development and customization of an interoperable infrastructure capable of harmonizing the disparate national implementations of EU-regulated reference processes in terms of communication protocols and data formats. Achieving seamless integration across business processes, which may be implemented differently across Member States, mandates the establishment of a uniform architecture. This architecture must empower process participants to configure the communication layer in a manner that ensures interoperability, while also upholding high levels of security and reliability across various use cases. Concurrently, the data exchange layer must remain aligned with prevailing and forthcoming standards and best practices concerning the underlying software frameworks and processes. The ultimate objective is to furnish a cohesive, consistent, and configurable foundation for communication protocols, primarily accessible and utilizable by "region connector" modules. By encapsulating the complexity of European energy data exchange within a structured and adaptable framework, the EDDIE architecture endeavors to streamline cross-border interactions and foster greater synergy among stakeholders across the energy landscape. This comprehensive approach not only facilitates efficient data exchange but also lays the Grant Agreement: 101069510 Dissemination level: PU Page 17 of 61 groundwork for future advancements in energy interoperability and collaboration on a panEuropean scale. Figure 1: Initial EDDIE Architectural Schema 2.3 State of the Art This section comprehensively explores the current landscape and historical context of energy data infrastructure and management, particularly focusing on the European Union but also drawing comparisons with the United States. 2.3.1 Legal Framework The transition to sustainable energy infrastructures is deeply intertwined with the accessibility and transparency of energy data. However, many countries lack the necessary Grant Agreement: 101069510 Dissemination level: PU Page 18 of 61 legal frameworks and standards to allow customers easy access to their energy data. This issue is especially pronounced in the findings of a 2022 survey by the University of Vienna, where international students struggled to retrieve their energy data from their home countries, with only 7 out of 42 attempts being successful [1]. This demonstrates the significant barriers customers face in accessing their energy data, which in turn impedes efforts to enhance energy efficiency and provide energy-related services on an international level. United States In the United States, there is no overarching federal regulation mandating the nationwide disclosure of energy data. Nevertheless, several states such as California, Columbia, and Illinois have enacted laws requiring metered data administrators to provide customers access to their energy information [2]. A notable initiative in this regard is the Green Button Initiative, introduced in 2012. Although voluntary, this initiative allows customers to download and share their energy usage information in a standardized format [3]. It has seen widespread adoption, with participation from over 50 utilities and electricity providers, thereby providing access to more than 60 million households. The success of the Green Button Initiative highlights the benefits of having a standardized format for energy data exchange across different suppliers and states. European Union The European Union has taken a proactive stance in ensuring the transparency and accessibility of energy data through the enactment of the Clean Energy Package in 2019 [4]. This comprehensive framework of laws and regulations aims to accelerate the shift towards a more sustainable energy ecosystem. The package includes directives that liberalize the European energy market and empower citizens to access and exchange their energy data easily [5]. Member states are required to develop transparent guidelines for managing and sharing energy-related data, including detailed metering and consumption data. The directive mandates that data must be made available to eligible parties in a fair and transparent manner, ensuring equal access to crucial information. Additionally, customers should not incur additional charges for accessing or sharing their data with eligible parties. The directive also encourages the installation of smart metering systems across the EU, allowing customers to request the installation of such systems to enhance energy efficiency and participation in the energy market. The broader Data Act Regulation (EU) 2023/2854 further supports these rights by stipulating that data holders must provide access to customer data in a standardized machine-readable format, extending this right to eligible third parties as well [6]. Grant Agreement: 101069510 Dissemination level: PU Page 19 of 61 2.3.2 State of the art in the EU The state of energy data infrastructure in the European Union is highly heterogeneous due to the diverse approaches adopted by different member states. While there is no single stateof-the-art solution for energy data access, the EU has provided a reference model through the Implementing Regulation (EU) 2023/1162 [7]. This regulation acts as a guideline for member states to implement data access rights for energy data and should in some shape or form be implemented by the member states. Data Exchange Models Energy data exchange models in the EU can be broadly categorized into centralized, decentralized, and hybrid exchanges [8]: 1) Centralized Data Exchanges In centralized exchanges, a single entity acts as both Metered Data Administrator (MDA) and Permission Administrator (PA), meaning it collects, stores, and distributes data, ensuring that all data is consistent and up-to-date. This entity is very often a central authority like a Transmission System Operator (TSO) or a Distribution System Operator (DSO). This model simplifies the process for third parties to access data as they only need to connect to one system. However, it can present a single point of failure, making it vulnerable to outages or attacks. 2) Decentralized Data Exchanges In decentralized exchanges, data management responsibilities are distributed among multiple entities. Each DSO acts as the MDA for their respective areas, making third-party access more complex as connections with multiple DSOs are required. This model avoids a single point of failure but can lead to inconsistencies and increased complexity in data access. 3) Hybrid Data Exchanges Hybrid exchanges combine aspects of both centralized and decentralized models. Typically, data is collected and stored in a decentralized manner but distributed centrally. This requires high standardization and cooperation among DSOs. Hybrid models aim to balance the benefits of centralized data access with the resilience of decentralized systems. Despite these classifications, what matters most to third parties is whether a country operates a Data Exchange Platform (DEP). A DEP facilitates the exchange of energy data between market participants, acting as a single access point for third parties. Countries without a DEP usually have a decentralized model, posing higher entry barriers for third parties. Figure 2 shows the state of centralization of data exchange models in Europe in 2016 as illustrated in [9]. As it can be seen, the majority of countries in Europe have a decentralized data exchange in place, but there is a trend towards implementing a DEP in the future. Grant Agreement: 101069510 Dissemination level: PU Page 20 of 61 Figure 2: State of centralization of data exchange models in Europe in 2016 [9]. Reference Model The EU's Implementing Regulation (EU) 2023/1162 [7] provides a reference model divided into six procedures, covering both near real-time data access and access to validated historical metering data (VHMD). Member states are required to map their national practices to this reference model and implement it by January 2025 [7]. The key procedures include: 1) Procedure 1: Access to validated historical metering and consumption data by the customer (process shown in Figure 3) • Customers identify their Data Access Provider (DAP), authenticate themselves, and request VHMD. • The DAP verifies the customer's identity and facilitates data transfer from the MDA to the customer. Grant Agreement: 101069510 Dissemination level: PU Page 21 of 61 Figure 3: Accessing VHMD by the customer [7]. 2) Procedure 2: Access to VHMD and consumption data by an eligible party (process shown in Figure 4) • Eligible parties must obtain customer permission through a Permission Administrator (PA) before accessing VHMD. • The PA manages permissions and ensures the transfer of data from the MDA to the eligible party. Grant Agreement: 101069510 Dissemination level: PU Page 22 of 61 Figure 4: Accessing VHMD by an eligible party [7]. Grant Agreement: 101069510 Dissemination level: PU Page 23 of 61 3) Procedure 3: Termination of service by an eligible party (process shown in Figure 5) • Eligible parties can terminate permissions, notifying the PA and the customer, who in turn ensures the MDA stops data transfer. Figure 5: The process of termination of service by an eligible party [7]. 4) Procedure 4: Revocation of Permission by the Customer (process shown in Figure 6) • Customers can revoke permissions using the PA, which then notifies the MDA and eligible party, stopping further data transfer. Grant Agreement: 101069510 Dissemination level: PU Page 24 of 61 Figure 6: The process of revocation of a permission by the customer [7]. Grant Agreement: 101069510 Dissemination level: PU Page 25 of 61 5) Procedure 5 & 6: Near Real-Time Data access (respective processes are shown in Figure 7 and Figure 8) • These procedures cover the activation and reading of near real-time data from smart metering systems, ensuring customers and third parties can access real-time data directly. Figure 7: Activate near real-time data flow from smart metering system [7]. Figure 8: Read near real-time data from smart meter or smart metering system [7]. Grant Agreement: 101069510 Dissemination level: PU Page 32 of 61 Figure 14: Sequence diagram of the Austrian process. Another very important part within the design and development process to find a suitable architecture for the message-based communication part in the interoperable communication layer for the Austrian case was the understanding how required data can be translated to respective CCMO XML messages that can be processed within the Ponton X/P messenger. Therefore, we have experimented with a translation service component built in directly in the EDA/Austria region connector. A possible integration of such a service in the respective region connector is depicted in Figure 15. There, the frontend uses the translation service to generate CCMO conform messages that can then be handled within the Ponton X/P messenger and be further transferred. An example for such a message translation is visible in Figure 16. In the sequence diagram in Figure 14 depicted above this translation service is integrated between the frontend and the “Ponton Adapter” as “Required Data to CCMO Translation Service”. In the following chapter 2.5, the general architecture that has been derived from the information of the country-specific workflow as described above is described in detail. Grant Agreement: 101069510 Dissemination level: PU Page 33 of 61 Figure 15: Inside view of the EDA Region Connector. Figure 16: Generation of a CCMO XML message. Grant Agreement: 101069510 Dissemination level: PU Page 34 of 61 2.5 General Architecture With the knowledge gathered during design and development phase in the first 18 months in the EDDIE project, we have been able to break this down to a composition of three distinct layers ((i) region connector layer, (ii) core layer, and (iii) application connector layer), as depicted in Figure 17. Figure 17: Layered concept of the EDDIE architecture. We have already foreseen the different messaging and communication aspects that are covered in the project’s work-package 4 and the respective tasks 4.2 (“message-based communication”), task 4.3 (“REST API-based communication”) and task 4.4 (“streamingbased communication”). The current document is mainly concerned with message-based communication, whereas the baseline for this has already been described in the previous section. In the following section 2.5.1 the three layers and their main purpose are described in detail. Grant Agreement: 101069510 Dissemination level: PU Page 35 of 61 2.5.1 Layered Architecture Concept 1) Region Connector Layer: A region connector is responsible for acquiring consumption data from a regional data sharing infrastructure and transforming it into a standardized data format based on the Common Information Model (CIM). To accommodate multiple regions, the region connector layer comprises various individual connectors, each tailored to manage the data transmission and permission processes specific to each region's data sharing infrastructure. 2) Core Layer: The core layer realizes the plugin architecture so that region connectors and application connectors can be added to the framework without affecting other plugins. Core components are available for handling permissions as well as the data streams from the region connectors. 3) Application Connector Layer: An application connector facilitates the transmission of consumption data to the eligible party's application by implementing a specific protocol. At present, there is one plugin that transmits JavaScript Object Notation (JSON) encoded CIM messages via Kafka. Nevertheless, the framework is designed for extensibility and can readily accommodate more efficient encoding formats, such as Protocol Buffers or Apache Avro. 2.5.2 Connection of other Messaging Solutions Moreover, the addition of support for other messaging solutions, such as Google Cloud Pub/Sub or Azure Service Bus, is particularly appealing for eligible parties with applications deeply integrated into proprietary cloud ecosystems. This flexibility ensures that the framework can adapt to various technical requirements and preferences. 2.5.3 Region Connector Architecture The primary requirements for inbound communication are encapsulated within the region connectors, which serve as the gateways for data flow into the EDDIE Framework. This is depicted in Figure 18. Grant Agreement: 101069510 Dissemination level: PU Page 36 of 61 Figure 18: Detailed Region Connector architectural concept. To facilitate message-based communication, some region connectors make use of middleware systems e.g., the Ponton X/P messenger is used for AS4 communication with EDA in Austria (see chapter 3 for a detailed description of the implementation of the Austrian case) and an MQTT broker is used for streaming communication with Administrative Interface for In-house Data Access (AIIDA) instances. While the region connectors standardize communication protocols and data formats, they also abstract the disparities among various permission administrators that are inherent to the regional data sharing infrastructure. This encapsulation ensures a consistent and secure interaction layer, despite the underlying heterogeneity. 2.5.4 Technical Implementation To facilitate ease of use, the EDDIE Framework is deployed as a monolithic application. This architecture not only simplifies operations but also enhances internal data transfer efficiency, thereby minimizing computational demands and reducing the energy footprint of the framework. Grant Agreement: 101069510 Dissemination level: PU Page 37 of 61 Monolithic systems, however, can be challenging to maintain. To address this, the EDDIE Framework's codebase is meticulously organized. The core of the framework realizes a plugin system built on the Spring Framework 3 . This design ensures that each region connector operates independently and interacts with other components exclusively through clearly specified interfaces. In the following chapter, the message-based communication is described in detail with adhering to the Austrian case as example for this particular communication and data exchange method. 3 See https://spring.io/projects/spring-framework Grant Agreement: 101069510 Dissemination level: PU Page 38 of 61 3 Message-Based Communication 3.1 Introduction The primary goal of work package 4 in the EDDIE project is to address the diverse methodologies of energy data exchange across Europe, ensuring interoperability and meeting stakeholder requirements for different data exchange types. This involves developing a customizable infrastructure to bridge the gap between various national implementations of EU-regulated reference processes, particularly in terms of communication protocols and data formats. Task 4.2 (“Message-based communication”) focuses on the traditional method of exchanging structured data sets, crucial for processes like supplier switching, nomination, and invoice data submission. The implementation will reuse existing technologies based on classical protocols like ebMS2.0 and AS4. Nevertheless, tasks 4.3 (REST API-based communication) and 4.4 (streaming-based communication) have already started within the project but are not subject of the current document. As already mentioned above, the architecture for the country specific regional connectors can have different architectures, due to e.g. legal constraints, energy data hubs, data models etc. Therefore, EDDIE has to be able to handle generic requests for different types of connectors. In general, the regional connector provides the main connection for the customer / EP to interact with the system (via an interface), to forward that information to the EDDIE core and to the database eventually, as well as to handle historic energy data from regional data hubs. For further details on the region connectors, please be referred to Deliverable 5.1 – “Intermediate report on best-practices and EU region connectors created” [29]. In the following the Austrian region connector will be presented in detail, since this incorporates the message-based communication approach and has been implemented as one of the first region connectors. Generally, the message-based communication is not restricted to Austria, the base functionality regarding message exchange is generic and can be applied within other regional connectors. Nevertheless, to showcase the complexity, the following section 3.2 provides detailed descriptions of the Austrian workflow. Grant Agreement: 101069510 Dissemination level: PU Page 39 of 61 3.2 Region Connector Austria 3.2.1 Introduction In Austria, where a decentralised, message-based technology (AS4) is used, eligible parties (EP) need to register at the EDA registration website [26], receive an EP identifier and then request a public/private key pair together with some connectivity information to communicate. Generally, EDA 4 can be thought of as the data access provider (DAP) for historically validated data in Austria. It works as a central messaging service for all the Distribution System Operators (DSOs) in Austria by implementing the AS4 messaging protocol and using the ebMS3 specification. The following Figure 19 (taken from the EDA website 5 ) illustrates this. Figure 19: Illustration of data exchange in Austria facilitated by EDA [26]. 4 See https://www.eda.at 5 See https://www.eda.at/wie-funktioniert-eda?lang=en Grant Agreement: 101069510 Dissemination level: PU Page 40 of 61 As it can be seen, instead of an API based approach, Austria exchanges energy data via encrypted XML files. How it works and relevant processes can be found at ebUtilities 6 . The relevant processes for EDDIE fall under the “Customer Consent Management” category and are described in the following sections. 3.2.2 EDA Processes Apart from being a data access provider (DAP), EDA in co-operation with Oesterreichs Energie 7 is also responsible for defining the business processes and market documents for the Austrian energy data exchange. The business processes and market documents are defined, consulted and published on ebUtilities.at. The relevant processes for project EDDIE all fall under the category of Customer Consent Management (CCM). As the name implies, these processes are about managing the permissions between the customers and eligible parties. The CCM processes can be further subdivided into two categories: 1) requesting customer data 2) terminating and revoking active permissions The process for requesting customer data is called CM_REQ_ONL. This process is complex but can broadly be simplified into two parts: 1) obtaining permission from the customer 2) the transmission of the requested data To simplify the explanation, the process has been split into two separate BPNM diagrams. The first diagram (Figure 20) depicts the steps related to obtaining the permission and the second diagram (Figure 21) shows the transmission of the data. Figure 21 also includes the processes that are related to terminating and revoking permissions. In other words, the second diagram shows a combination of the CM_REQ_ONL process with the termination (CM_REV_SP) and revocation (CM_REV_CUS and CM_REV_IMP) processes. This makes it easier to understand the relationship between these processes, as termination and revocation can happen at any point after permission has been given but data transmission has not yet been completed. 6 See https://www.ebutilities.at/prozesse 7 See https://oesterreichsenergie.at/ Grant Agreement: 101069510 Dissemination level: PU Page 41 of 61 Figure 20: BPMN diagram showing steps for obtaining the permission. The process starts with the customer informing the eligible party (EP) that they want to share their data (1). The EP then creates and sends a CMRequest message with message code ANFORDERUNG_CCMO (see Figure 22 for an example of such an XML message) for the data the final customer wants to share to the DSO (2). The DSO will then validate the request (3). If the request is invalid, the DSO will send a CMNotification message with message code ABLEHNUNG_CCMO to the EP (4.b) at which point the process ends. If the request is valid, the DSO will send a CMNotification message with message code ANTWORT_CCMO to the EP (see Figure 23 for an exemplary message) and make it possible for the final customer to view and accept the request online (4.a). The process does not specify how the request is presented to the customer. However, it is typically displayed directly in the customer portal of the DSO if the metering point of the final customer was included in the CMRequest message. If the metering point was not included, the final customer has to provide the cmRequestId to the DSO to view and accept the request, this requires that the EP informs the final customer about the cmRequestId. Once the final customer can view the request, they can either accept (5.a) reject (5.b) or ignore (5.c) the request. If the final customer rejects or ignores the request, the DSO will send a CMNotification message with message code ABLEHNUNG_CCMO to the EP (6.b) and the process ends. If the final customer accepts the request, the DSO will send a CMNotification message with message code ZUSTIMMUNG_CCMO to the EP (6.a) and schedule the data transmission (7). Grant Agreement: 101069510 Dissemination level: PU Page 48 of 61 metering device could support multiple meterCodes and therefore a ConsumptionRecord message could contain multiple EnergyData objects with different meterCodes for the same metering point. A typical ConsumptionRecord message contains one Energy object with one EnergyData object and depending on the resolution of the data multiple EnergyPoint objects. The DSOs usually aggregate requests for daily measurements into a single ConsumptionRecord message, while requests for quarter-hourly measurements are sent as one ConsumptionRecord message per requested day in the time range where each message contains 96 EnergyPoint objects. Figure 28: Simplified version of the MasterData message. Figure 28 shows a simplified version of the MasterData message. The MasterData message contains meta information about the metering point and the final customer. Information about the capabilities of the metering device is represented by a MeteringPointData object. The deliveryAddress stores the physical address of the metering point. The contractPartner identifies the final customer who is associated with the metering point. The billingData refers to information about how the final customer is billed. If the invoice is sent to a different address than the metering point or the contract partner is a company, the invoiceRecipient object will contain the address of the recipient, otherwise, it will not be present. Grant Agreement: 101069510 Dissemination level: PU Page 49 of 61 3.2.4 Setup/Onboarding For the AS4 communication with EDA, the Ponton X/P Messenger is used. Generally, the region connector for Austria uses the Ponton X/P messenger to communicate with the DSOs. The communication with the Ponton X/P Messenger itself works via their Java API which uses a Websocket connection in the background. As for future solutions, it is considered implementing the AS4 protocol or apply available open-source implementations. 3.2.5 PONTON X/P Messenger The PONTON X/P Messenger is an essential building block for connecting the EDDIE Framework to the EDA partners. As the usage of Ponton X/P and AS4 protocols is specific to the Austrian data sharing infrastructure, the connection to the EDDIE Framework is implemented as part of the EDA region connector for Austria. AS4 offers significant advantages for B2B integration including interoperability, secure data transmission, reliability and non-repudiation of messages. These benefits are realized through the exchange of standardized, encrypted, and signed messages. Non-repudiation guarantees that once a message is sent, the sender cannot deny its submission or authenticity, which is crucial for many business processes in the energy sector. Grant Agreement: 101069510 Dissemination level: PU Page 50 of 61 Figure 29: Illustration of the message-based communication via Ponton X/P messenger in AT. The communication and permission components of the EDA region connector connect to the PONTON X/P messenger using the Adapter library. This handles communication with the Ponton X/P messenger which is deployed as a separate software system on the eligible party’s infrastructure. PONTON X/P fully implements the AS4 standard, incorporating all its benefits. Additionally, PONTON X/P enriches the functionality with certificate management and partner management features. The Ponton X/P messenger serves as an intermediary by translating AS4-based market communications into formats compatible with the internal software systems used by market participants. These organizations may connect to the Ponton X/P messenger using a variety of protocols, such as REST, SOAP, direct File System access, database access or by employing the Ponton X/P Adapter library. Grant Agreement: 101069510 Dissemination level: PU Page 51 of 61 3.3 Exemplary Message-based Request As this deliverable is classified as being of type “DEM – Demonstrator, pilot, prototype”, this section presents an actual workflow of a message-based data request. For details on the permission façade, the information schema or the details on the region connectors, please be referred to the respective deliverables “D2.2 – Information schema defined” [31], “D3.1 – EDDIE Consent façade” [30] and “D5.1 – Intermediate report on best practices and EU region connectors created” [29]. First, the final customer uses the permission façade and the respective frontend on the EPs website to select the correct country, permission administrator and enters his/her metering point id, and clicks on the button “Connect” as shown in Figure 30. Figure 30: Entering the customer related data. After the click on the button, the consent request ID is generated and displayed on order to be able to identify this request internally. A corresponding ANFORDERUNG_CCMO message is created and transmitted and the status set to SENT_TO_PERMISSION_ADMINISTRATOR. The information is shown on the EDDIE frontend, as it is depicted in Figure 31. Grant Agreement: 101069510 Dissemination level: PU Page 52 of 61 Figure 31: Information of the request on the EDDIE frontend The corresponding ANTWORT_CCMO message is shown in Figure 32 – please note the matching request ID. Figure 32: Generated ANTORT_CCMO message. The final customer is now able to accept or decline the request by logging into his/her DSO portal. In our exemplary case, this is “Netz OÖ” 10 – this request is shown in Figure 33. 10 See https://www.netzooe.at Grant Agreement: 101069510 Dissemination level: PU Page 53 of 61 Figure 33: Permission request in the customers DSO portal. After the final customer clicks on “accept”, the final customer must accept the terms (see Figure 34) and after that, the permission is granted. Figure 34: Declaration of consent in the user's DSO portal. After granting the permission, the status is changed to fulfilled and the request is completed. This information is shown on the EDDIE frontend as depicted in Figure 35. Grant Agreement: 101069510 Dissemination level: PU Page 54 of 61 Figure 35: Illustration of the information of a fulfilled request. Internally, this is represented by an AS4 message with message code ZUSTIMMUNG_CCMO as shown in Figure 36. Figure 36: Illustration of a ZUSTIMMUNG_CCMO message. Since this exemplary request was about validated historical data, the corresponding AS4 message can be immediately sent. A clipping of this message is shown in Figure 37. Grant Agreement: 101069510 Dissemination level: PU Page 55 of 61 Figure 37: Clipping of a DATEN_CRMSG message. Additionally, the complete message protocol for this request can be seen in the Ponton X/P message backend, as shown in Figure 38. Starting with the first message 16407 – ANFORDERUNG_CCMO – to message 16415 – DATEN_CRMSG – the whole bunch of AS4 messages can be tracked. Figure 38: Message log in the Ponton X/P backend. Grant Agreement: 101069510 Dissemination level: PU Page 56 of 61 4 Outlook & Roadmap This deliverable mainly dealt with tasks 4.1 (“Communication layer design”) and 4.2 (“Message-based communication”) of the EDDIE grant agreement. Nevertheless, other ways of communication need to be incorporated to cover the broad spectrum of different European solutions. As foreseen in the grant agreement, two major communication methods are covered, namely REST API-based communication and streaming-based communication. At the time of writing this current document, both tasks have already started and these communication methods have already been implemented to some extent for countryspecific region connectors. For example, Figure 39 and Figure 40 show REST clients and REST API implementations for the French region connector incorporating Enedis. Figure 39: Illustration of the French Enedis REST Client. Also, streaming-based communication utilizing Apache Kafka 11 as streaming platform is already built in the EDDIE framework. Granted request and exchanged data can be 11 See https://kafka.apache.org Grant Agreement: 101069510 Dissemination level: PU Page 57 of 61 consumed via Kafka. Additionally, we are also considering other protocols (like MQTT) and generally different communication messages. Figure 40: First implementation attempts against the Enedis REST API. Details on further approaches for the interoperable communication layer within EDDIE’s workpackage 4 will be covered in deliverable “D4.2 – Three different approaches for the communication layer are implemented”. This deliverable is due in month 30 (which would be June 2025).