D3.2 Final Architecture And Interface

DEVICE-EDGE-CLOUD INTELLIGENT COLLABORATION FRAMEWORK Grant Agreement: 101092582 D3.2 Final Architecture And Interface This project has received funding from the European Union’s Horizon Europe Research and Innovation Programme under Grant Agreement No 101092582. D3.2 Final Architecture And Interface 2 Document Information Deliverable number: D3.2 Deliverable title: Final Architecture And Interface Deliverable version: 1.0 Work Package number: WP3 Work Package title: Open Framework and Virtual Training Environment Responsible partner GWDG Due Date of delivery: 2024-11-30 Actual date of delivery: 2024-11-29 Dissemination level: PU Type: R Editor(s): Felix Stein (UGOE) Mirac Aydin (GWDG) Reviewer(s): Dr. Sachin Nanavati (NAG) Aasish Kumar Sharma (UGOE) Project name: Device-Edge-Cloud Intelligent Collaboration framEwork Project Acronym: DECICE Project starting date: 2022-12-01 Project duration: 36 months Rights: DECICE Consortium ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 3 Document History Version Date Partner Description 0.1 2024-11-22 UGOE/GWDG First Version of Deliverable 0.2 2024-11-25 NAG/UGOE Internal Review 1.0 2024-11-28 UGOE/GWDG Feedback Implementation and Finalization Acknowledgement: This project has received funding from the European Union’s Horizon Europe Research and Innovation Programme under Grant Agreement No 10192582. Disclaimer: The content of this publication is the sole responsibility of the authors, and in no way represents the view of the European Commission or its services. ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 4 Executive Summary This deliverable provides an overview of the final architecture of the DECICE framework, including its interface and key components. The document gives a detailed explanation of each component, their functionality, and how they interact with each other. Additionally, the deliverable outlines the API specifications within the framework, providing guidance on how to access relevant documentation. The security measures implemented within the framework and Kubernetes are also discussed, with a focus on data storage, data transfer, and user management. This includes an examination of the protocols to ensure the secure deployment and integration of the framework with external services. Furthermore, this deliverable provides insight into the deployment process of the DECICE framework, highlighting the steps taken to ensure seamless integration with external services. The document also acknowledges the challenges encountered during the development process and presents a roadmap for future enhancements and improvements. ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 5 Contents 1 Purpose and Scope of the Deliverable 6 2 Abstract / publishable summary 6 3 Project objectives 7 4 Changes made and/or difficulties encountered, if any 7 5 Sustainability 7 6 Dissemination, Engagement and Uptake of Results 7 6.1 Targetaudience ..................................... 7 6.2 Record of dissemination/engagement activities linked to this deliverable . . . . . . . 8 6.3 Publications in preparation OR submitted . . . . . . . . . . . . . . . . . . . . . . . 8 6.4 Intellectual property rights resulting from this deliverable . . . . . . . . . . . . . . . 8 7 Detailed report on the deliverable 8 7.1 Architectural Refinements and Progress . . . . . . . . . . . . . . . . . . . . . . . . 9 7.2 Overview of System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 7.3 API Specification and Documentation . . . . . . . . . . . . . . . . . . . . . . . . . 14 7.4 SecurityMeasures.................................... 17 7.4.1 DECICEFramework............................... 17 7.4.2 Kubernetes ................................... 19 7.5 Deployment and Interoperability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 7.6 PlannedAdvancements ................................. 21 8 Summary 22 A Appendix 23 A.1 ControlManager..................................... 23 A.1.1 Routes...................................... 23 A.1.2 Route Specification - Example . . . . . . . . . . . . . . . . . . . . . . . . . 24 A.1.3 Schemas..................................... 25 A.2 PromQLWrapper.................................... 25 A.2.1 Routes...................................... 25 A.3 PSGC .......................................... 25 A.3.1 Routes...................................... 25 A.3.2 Schemas..................................... 25 A.4 DigitalTwin....................................... 26 A.4.1 Routes...................................... 26 A.5 SchedulerController................................... 26 A.5.1 Routes...................................... 26 A.6 Scheduler ........................................ 27 A.6.1 Routes...................................... 27 ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 6 1 Purpose and Scope of the Deliverable The primary objective of this deliverable is to provide a comprehensive overview of the finalized architecture of the DECICE framework. Additionally, this document includes detailed explanations of the framework’s API specification, related documentation, and implemented security measures. A summary of the framework’s deployment is also provided. Furthermore, potential future enhancements to the framework are outlined. 2 Abstract / publishable summary This deliverable presents the finalization of the DECICE framework’s architecture and APIs, which have undergone significant improvements and refactoring since the previous deliverable, D3.1 - Synthetic Test Environment. To ensure the secure transfer of critical data through the framework’s layers, an OAuth2 authentication mechanism has been implemented, enhancing security, user access, and reliability. Furthermore, the combination of HTTPS, TLS 1.3, and Role-Based Access Control has been employed to secure data transfer. On the Kubernetes side, RBAC, TLS certification, and hierarchical namespaces have been implemented to facilitate secure communication between pods and prevent unauthorized access to user data. To facilitate easy deployment, the developed components have been containerized, and Helm Charts have been created for each component. Additionally, GitLab CI/CD pipelines have been utilized to manage the deployment process. The interoperable components of DECICE have enabled seamless integration with external services, such as Prometheus and Grafana. This deliverable also discusses the challenges encountered and outlines future steps, including planned enhancements such as a Software Development Kit (SDK) and web interface. The primary outcome of this work is the finalization of the architecture, which enables easy enhancement of the framework in terms of development and deployment. This achievement provides a solid foundation for future development, allowing for more efficient and effective integration of new features. ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 7 3 Project objectives This deliverable contributes directly and indirectly to the achievement of all the macro-objectives and specific goals indicated in section 1.1.1 of the project plan: Macro-objectives Contribution of this deliverable (O1) Develop a solution that allows to leverage a compute continuum ranging from cloud and HPC to edge and IoT. This deliverable provides an overview of the overall DECICE framework architecture, highlighting its efficient management of the hybrid compute continuum. (O2) Develop a scheduler supporting dynamic load balancing for energy-efficient compute orchestration, improved use of green energy, and automated deployment. This deliverable demonstrates the integration of the Integrated AI Scheduler with the Production Environment and the Virtual Training Environment (VTE). (O3) Design and implement an API that increases control over network, computing and data resources. This deliverable details the development and functionality of various APIs within the DECICE framework, providing insight into their operational mechanics. (O4) Design and implement a Dynamic Digital Twin of the system with AI-based prediction capabilities as integral part of the solution. This deliverable outlines the deployment processes for integrating developed components within the DECICE framework into the hybrid compute continuum. (O5) Demonstrate the usability and benefits of the DECICE solution for real-life use cases. This deliverable shows the DECICE framework’s capabilities in providing a unified management layer for the hybrid compute continuum. (O6) Design a solution that enables service deployment with a high level of trustworthiness and compliance with relevant security frameworks. This deliverable describes the security measures and methodologies used within the DECICE framework itself, as well as those securing the hybrid compute continuum. 4 Changes made and/or difficulties encountered, if any No significant changes to the project plan were made. No significant challenges were encountered during implementation. 5 Sustainability Design and optimization of components in the project are tightly coupled to multiple WPs such as WP2, WP3 and WP4. Every partner in each work package will communicate their result regularly for an optimal integration of each component into the framework. 6 Dissemination, Engagement and Uptake of Results 6.1 Target audience As indicated in the Description of the project, the audience for this deliverable is: ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 8 ✓The general public (PU) The project partners, including the Commission services (PP) A group specified by the consortium, including the Commission services (RE) This report is confidential, only for members of the consortium, including the Commission services (CO) 6.2 Record of dissemination/engagement activities linked to this deliverable See Table 1. Type of dissemination and communication activities Details Date and location of the event Type of audience activities Zenodo Link Estimated number of persons reached None N/A N/A N/A N/A 0 Table 1: Record of dissemination / engagement activities linked to this deliverable 6.3 Publications in preparation OR submitted See Table 2. In preparation or submitted? Title All authors Title of the periodical or the series Is/Will open access be provided to this publication? None N/A N/A N/A N/A Table 2: Publications related to this deliverable 6.4 Intellectual property rights resulting from this deliverable None. 7 Detailed report on the deliverable This deliverable provides a comprehensive and systematic overview of the finalized architecture and interfaces for the DECICE framework. Since the publication of our last deliverable, significant progress has been made in refining the system’s architecture and enhancing its capabilities. These advancements are detailed in the following sections, offering insights into how the system has evolved to address the technical and functional requirements of the project. ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 9 The first section begins with a discussion of the architectural refinements and progress achieved since the previous deliverable. This section highlights the most notable changes made to the DECICE system, focusing on improvements that have enhanced the overall design, functionality, and efficiency. The next section presents a thorough overview of the system architecture, providing a detailed breakdown of each component and microservice. It breaks down each component and microservice, explaining their specific roles and how they interact with one another within the framework and training environment to form a cohesive and efficient system architecture. The subsequent section focuses on the API specification and documentation, detailing the user facing endpoints that facilitate interaction with the DECICE framework and training environment, as well as the internal routes that enable seamless communication between microservices. The section also discusses the use of the HTTP protocol to implement RESTful services, the adoption of OpenAPI for a thorough documentation, and the mechanisms in place for authentication and authorization to ensure secure and reliable user interactions. The Integration and Interoperability section examines how the DECICE framework integrates with external services, including the compute plane for job submission and computation, as well as monitoring tools like Prometheus and Grafana. It also describes the deployment strategies employed, such as containerization and the use of Kubernetes for orchestrating the framework and compute plane. Lastly, the report discusses the challenges encountered during development and outlines potential areas for future enhancement. While the architecture has been finalized, ongoing efforts will focus on improving usability, user-friendliness, accessibility, and security in upcoming development sprints. 7.1 Architectural Refinements and Progress Since the initial architecture outlined in D3.1 - Synthetic Test Environment, significant changes and advancements in terms of architectural refinement have been made in the development of the DECICE framework in order to enhance development speed, scalability and security. One of the most notable advancements is the introduction of a central managing instance for both - the virtual training (VTE) as well as the production environment (PE). Since the design and development of the VTE began earlier than the production environment, the VTE was already in a beta state when the development of the actual DECICE framework began. In design workshops within work package 3 (WP3) we realized that the initial design of the VTE fits the requirements of the production environment quite well and only minor adaptions were needed to start the development process. In return, observations we did when writing the production instance led to significant improvements for the VTE which we had not yet put into consideration. This back and forth development cycle ultimately led to a unification of both instances. The VTE controller was encapsulated and refactored into the more general controlling instance DECICE Control Manager (CM) and also integrated in the production framework. Both CMs now have the same capabilities in terms of user management, job scheduling, telemetry observation and security measures with only minor differences in route calling of different microservices. One example being the provision of a job or multiple jobs: while the PE allows for job uploading and scheduling on real systems in a heterogeneous compute plane, the VTE allows for ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 16 OpenAPI9for our documentation, a specification and standard for building, describing, and consuming RESTful APIs. Each service provides routes that can be called to retrieve, send or alter different information, depending on the capabilities and tasks of each service. These operations happen in the form of: GET, POST, UPDATE, DELETE that are being send between services. An example documentation is depicted in Figure 3. It shows an excerpt of the Control Managers routes for handling users, it allows for creating and deleting users as well as updating retrieving user information. Figure 3: DECICE Control Manager user handling These routes are part of the broader API spectrum we implemented in the CM and are part of Task T3.2: DECICE API, the DECICE APIs (D-APIs). The D-APIs consist of four key APIs handling job and user specific tasks, telemetry tasks, security and the overall control mechanism of the scheduler and the platform as a whole. •Control & Security (Administrative API): Enables administrators to manage users, configure settings, and issue commands to the underlying platform. •Login (Authentication Endpoint): Accepts username and password combinations from users, returning a scoped token upon successful login. This process leverages OAuth2 workflows. •Job Management (Job Submission & Inquiry): Allows authenticated users to submit new jobs or retrieve information about existing jobs, with submission permissions restricted to projects for which they have authorized access. •Telemetry (Metrics Access): Provides users with telemetry data, filtered according to their permission levels. Metrics can be retrieved via existing API routes that execute PromQL queries on the backend or by directly submitting PromQL queries. The OpenAPI documentation is useful for developers and users of an SDK alike. It not only provides insights into what routes to call to retrieve a specific information, it also specifically states how the request should like and provides an example upon a successful or bad request. An outlook of how the documentation looks like is further depicted in Appendix A. Product Documentation Besides the technical documentation of the DECICE framework and its functionalities there is also a product documentation available which we are also using internally as a team documentation. This has two benefits. It allows new developers to maneuver themselves through the development process more quickly and serves as a central point of entry to understand the usage of the system. It is 9OpenAPI Specification - https://www.openapis.org/ ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 17 also the foundation for a proper product documentation once the applications launches to give end users the possibilities to quickly find their way through the framework. Figure Figure 4 depicts our current product documentation, the DECICE Knowledge Base. Figure 4: DECICE Knowledge Base It offers in-depth insights into the architectural documentation, JSON structures of the job submission process, DB schemas and dependencies necessary for a successful deployment of the whole application as well as tutorials for setting up and configuring dependencies for the compute plane such as Rook, Ceph or BeeGFS depending on the storage solution that will be picked. Properly written README files which are often used by users to get a first idea on how an application works when scrolling through GitHub or GitLab repositories are also placed in the repository, complementing the user documentation for the DECICE framework. 7.4 Security Measures In this section, we detail the security measures implemented within the project. These measures are designed to ensure the confidentiality, integrity, and availability of data and resources. The discussion is divided into two key areas: the DECICE Framework, which represents the foundational aspects of the project’s architecture, and Kubernetes, the container orchestration platform used to manage workloads and deployments securely. 7.4.1 DECICE Framework Security measures are critical to ensure the trustworthiness, reliability, and resilience of the DECICE Framework and its operations. In an environment where sensitive data flows through multiple layers - spanning edge devices, cloud platforms, and containerized infrastructures - security safeguards are necessary to protect against unauthorized access, data breaches, and malicious attacks. Without robust security measures, the framework would be vulnerable to threats like data theft, service disruptions, and regulatory non-compliance, which could undermine stakeholder confidence and jeopardize project success. In the development process of the DECICE framework we focused on three critical areas: data transfer, data processing and data storage. Each of these components plays a vital role in ensuring the confidentiality, integrity, and availability of the system ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 18 Data Transfer Securing data transfer is a critical aspect of safeguarding sensitive information as it moves across networks, especially in distributed and interconnected environments. However, several vulnerabilities can compromise the confidentiality, integrity, and authenticity of data in transit. These vulnerabilities include unencrypted transmission, which leaves data exposed to interception; weak protocols that fail to provide adequate protection against evolving cyber threats; insecure APIs that can be exploited to access or tamper with data during transfer; and inadequate endpoint verification, which may allow malicious actors to masquerade as legitimate systems. These vulnerabilities expose data transfer to significant threats. Man-in-the-middle attacks can intercept and manipulate data in transit, replay attacks can use captured transmissions to gain unauthorized access, and eavesdropping can reveal sensitive information to attackers. Additionally, improperly secured data transfer can fall victim to traffic analysis, where even encrypted traffic is analyzed to infer sensitive details about the communication. To mitigate these vulnerabilities and address potential threats, we have implemented a robust suite of security measures. All data transfers are secured using HTTPS and TLS 1.3, which provides strong encryption and eliminates outdated cryptographic vulnerabilities, making the communication between the user and the server running the DECICE framework secure. Additionally, an RBAC system integrated with OAuth 2.0 protocols is employed to ensure that only authenticated and authorized entities can initiate data transfers and access the platform, adding an extra layer of control. To fully use the platforms capabilities users need to register and setup an account first, providing a strong password during the registration process. To further enhance security, we deploy mutual authentication between endpoints, ensuring that all communication occurs only between verified parties. To ensure continuous protection and rapid response to anomalies, we have integrated extensive logging within the framework to be able to collect and display the logged information using the monitoring capabilities using Grafana. This system collects and visualizes real-time data on data transfer activities, providing detailed insights into the flow of information across the network. Data Processing Ensuring the security of data processing is critical, as this stage is vulnerable to several risks that can compromise the integrity and reliability of the system. Key vulnerabilities include untrusted code execution, where data processed with unverified scripts or third-party libraries could introduce malicious behavior, and insufficient input validation, which leaves the system open to injection attacks, such as SQL injection. Additionally, excessive permissions in processes that do not follow the principle of least privilege expose sensitive data unnecessarily, while resource exhaustion vulnerabilities allow attackers to overload system resources through malicious inputs, potentially leading to service disruption. To mitigate these risks, we have implemented a robust set of best practices. All inputs are thoroughly validated and sanitized before processing, reducing the risk of injection attacks. Only trusted and verified libraries or scripts are used to ensure secure and authenticated code execution. We also used a set of ORM layers abstracting database reads and writes to prevent code injection that could enable attackers to execute arbitrary code during processing, potentially accessing or modifying ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 19 sensitive data. To enhance isolation and containment, containerization technologies such as Docker are employed, creating secure environments where processes are isolated from each other, preventing potential breaches from spreading. Regular security audits are conducted to identify and eliminate permission creep, ensuring that access and permissions strictly follow the principle of least privilege. Data Storage Securing data storage is fundamental to protect sensitive information and ensuring the integrity and availability of the system. However, several vulnerabilities can expose stored data to significant risks. Unencrypted storage leaves data accessible if the medium is compromised, while weak access controls fail to adequately enforce user roles and permissions, increasing the risk of unauthorized access. To address these vulnerabilities and threats, we implement a multi-layered approach to secure data storage. All data within the framework is stored across different databases. User data is stored in a SQL database, ensuring a clear separation of critical information. Furthermore, user passwords are encrypted with a hash and a salt, providing an additional layer of security. Since SQL databases are not encrypted by default, when rolling out a production deployment of the framework we also need to take care of incorporating an encryption to further protect the stored data. For this we are looking into data encryption using AES-256 or other strong encryption algorithms, ensuring that sensitive information remains protected even if the storage medium is compromised. We enforce strict RBACs to regulate permissions and limit access to authorized users only. In cloud environments, robust storage security measures are implemented, including correct configurations, continuous monitoring, and strong identity management practices. In this context we can rely on already proven solutions and implementations like BeeGFS for HPC systems and Ceph or Lustre for cloud storage solutions managed by Kubernetes. 7.4.2 Kubernetes Within the DECICE framework, Kubernetes serves as the primary orchestration platform, playing a crucial role in managing operations across the compute continuum. This includes deploying jobs and pods, creating Persistent Volumes (PV) for data storage and access, and managing all communications between components. As a result, securing communication between pods and nodes is extremely important. Furthermore, effective user permission management is essential for maintaining data security, protecting data transfer, and preventing unauthorized access and potential security breaches. Role-Based Access Control (RBAC) In Kubernetes, RBAC is a key security control to ensure that cluster users and workloads have only the access to resources required to execute their roles. RBAC defines two types of roles: ClusterRoles and Roles. ClusterRoles are used to grant access to cluster-wide resources, while Roles are used to grant access to resources within a namespace, which provides a mechanism for isolating groups of resources within a single cluster. By defining roles and assigning them to users or service accounts, administrators can restrict access to sensitive resources and ensure that users can only perform ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 20 actions that are necessary for their tasks. In DECICE, RBAC is implemented to manage user permissions and maintain data security. For example, a developer role might be granted read-only access to pods, while an administrator role might be granted full access to manage and update pods. This implementation helps to prevent unauthorized access and potential breaches, aligning with the framework’s focus on user permissions, data security and secure data transfer. Transport Layer Security (TLS) TLS certifications has an important role in securing communication between components, such as pods, services, and the API server. TLS is a cryptographic protocol that provides end-to-end encryption for data ensuring that data remains confidential. In Kubernetes, TLS certifications are used to authenticate and verify the identity of components, preventing attacks and ensuring that only authorized components can communicate with each other. The API server, etcd, and pod-to-pod communication are all secured using TLS certifications. TLS certifications are essential for maintaining the security and integrity of the hybrid compute plane in DECICE. By using TLS certifications, DECICE ensures that all communication between pods and nodes is encrypted and authenticated, preventing unauthorized access and data breaches. This is particularly important for DECICE, as it handles sensitive data and requires a high level of security to protect it. Hierarchical Namespaces Hierarchical Namespaces is a feature that allows administrators to organize and manage resources in a hierarchical structure. This feature enables the creation of nested namespaces, where a parent namespace can contain multiple child namespaces. Each namespace can have its own set of resources, such as pods, services, and deployments, and can be managed independently. Hierarchical Namespaces provides a flexible and scalable way to manage complex environments, making it easier to organize and secure resources. This feature is particularly useful in multitenant environments, where multiple organizations or departments share the same Kubernetes cluster. In DECICE, Hierarchical Namespaces provides a powerful way to manage security and access control across the compute continuum. One of the key advantages of Hierarchical Namespaces is that security rules, such as RBAC policies, can be applied from the parent namespace to child namespaces, ensuring that access controls are consistently enforced across the entire namespace hierarchy. This means that administrators can define a set of security policies at the parent namespace level, and have them automatically applied to all child namespaces, reducing the administrative workload and ensuring that security is consistently enforced. Additionally, Hierarchical Namespaces allows DECICE to take advantage of inheritance, where child namespaces can inherit resources and policies from their parent namespace, making it easier to manage and scale the environment. ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 21 7.5 Deployment and Interoperability During the development of the DECICE framework, a microservice architecture was adopted, enabling developers to create services and components independently. These developed services are then containerized and prepared for deployment on Kubernetes, taking advantage of the platform’s built-in scalability and management capabilities. However, manual container deployment is highly inefficient. To address this challenge, the DECICE framework utilizes Helm charts10, generating template files that facilitate easy deployments on Kubernetes while enabling effortless application version tracking. Additionally, the framework is integrated with GitLab CI/CD pipelines, ensuring a smooth development-to-production workflow and improving overall administration efficiency. The DECICE framework is designed with interoperability at its core, achieved through specialized components that bridge diverse services, compute planes, and platforms. For example, the PromQLto-JSON Wrapper connects to external Prometheus instances, pulling metrics without requiring a new Prometheus server installation, thus leveraging existing monitoring infrastructures. Moreover, a dedicated Grafana Helm chart is included for easy deployment and customized dashboard creation for administrators and users. The PSGC further ensures effective communication between the CM and Kubernetes-deployed compute planes, enabling seamless job submissions, resource allocation, command execution, and data uploads by effectively overcoming architectural, implementation, and technological differences. 7.6 Planned Advancements Although the final architecture of the DECICE framework has been completed, there are still several key components that need to be integrated and refined to ensure smooth functionality. Firstly, a user-friendly web frontend will be implemented to provide a graphical interface for end-users to interact with the framework. This web frontend will leverage the external-facing APIs offered by the DECICE Control Manager, enabling users to submit jobs, view job status, retrieve metrics, and log in. This integration will be particularly beneficial for less experienced users, who will be able to navigate the system backed by a graphical user interface. Secondly, a comprehensive Software Development Kit (SDK) will be developed to facilitate the creation of DECICE API clients. By providing libraries and tools, the SDK will enable other developers and organizations to build their own applications and integrate them with DECICE, thereby expanding the framework’s capabilities and ecosystem. Thirdly, the Snakemake11 workflow management system will be integrated. It is a tool for creating reproducible and scalable data analyses. Workflows are defined using an easy-to-read, adaptable, yet powerful specification language built on top of Python. The main purpose of integrating Snakemake is to enable users to express complex logic in a human-readable and self-contained way, and to scale their workflows on Kubernetes easily. Lastly, the data upload mechanism will undergo improvements to prevent potential overloads on the DECICE Control Manager and ensure secure data flow within the framework. This optimization will 10Helm - https://helm.sh/ 11Snakemake - https://snakemake.github.io/ ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 22 be crucial in maintaining the system’s performance and reliability. In addition to these enhancements, the framework will undergo iterative optimization and refinement without altering its overall architecture until the project’s completion. This ongoing improvement process will ensure that the framework remains efficient, scalable, and adaptable to evolving requirements. 8 Summary This deliverable presented a comprehensive overview of the finalized architecture and interfaces of the DECICE framework. Significant architectural refinements have been made since the previous deliverable, including the unification of the Control Manager for both the Virtual Training Environment and the Production Environment. This unification enhances development speed, scalability, and security, allowing changes to be reflected automatically across both environments. We detailed the system architecture, highlighting key components such as the Control Manager, Platform Specific Glue Code, Compute Continuum, Digital Twin, Scheduler Controller, Metric Storage, PromQL-to-JSON Wrapper, and Integrated AI Scheduler. Each component’s functionality and interactions were explained to provide a clear understanding of the framework’s operations. The API specifications and documentation were outlined, emphasizing the use of OpenAPI for comprehensive documentation and the implementation of RESTful services using HTTP protocols. Security measures were discussed in depth, focusing on data transfer, data processing, and data storage. Implementations include OAuth2 authentication, HTTPS, Role-Based Access Control, and best practices like input validation and the principle of least privilege to ensure the framework’s trustworthiness and compliance with relevant security frameworks. Deployment strategies were presented, highlighting the use of containerization, Helm charts, and GitLab CI/CD pipelines for efficient deployment and management of the framework. The interoperability of the DECICE framework with external services like Prometheus and Grafana was also addressed, showcasing seamless integration capabilities. While the architecture has been finalized, the deliverable acknowledges challenges encountered and outlines future enhancements. These include the development of a user-friendly web frontend to improve accessibility for end-users, a comprehensive Software Development Kit (SDK) to facilitate the creation of DECICE API clients, the Snakemake integration and improvements to the data upload mechanism to prevent potential overloads and ensure secure data flow. ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 23 A Appendix A.1 Control Manager A.1.1 Routes ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 24 A.1.2 Route Specification - Example ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582 D3.2 Final Architecture And Interface 25 A.1.3 Schemas A.2 PromQL Wrapper A.2.1 Routes A.3 PSGC A.3.1 Routes A.3.2 Schemas ©2022 DECICE Horizon Europe |HORIZON-CL4-2022-DATA-01-02 |101092582