D1.7 EMERALD Integrated solution - v1

Deliverable D1.7 EMERALD Integrated solution – v1 Editor(s): Iñaki Etxaniz Responsible Partner: TECNALIA Research & Innovation Status-Version: Final-v1.0 Date: 30.04.2025 Type: Other (SW) Distribution level (SEN, PU): PU D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 2 of 46 www.emerald-he.eu Project Number: 101120688 Project Title: EMERALD Title of Deliverable: EMERALD Integrated solution – v1 Due Date of Delivery to the EC 30.04.2025 Workpackage responsible for the Deliverable: WP1 - Concept and methodology of EMERALD Editor(s): Iñaki Etxaniz (TECNALIA) Contributor(s): FABA, TECNALIA, Fraunhofer, CNR, SCCH Reviewer(s): Nico Haas (Fraunhofer) Cristina Martínez, Juncal Alonso (TECNALIA) Approved by: All Partners Recommended/mandatory readers: WP1, WP2, WP3, WP4, WP5 Abstract: Initial integrated solution of the EMERALD audit suite Keyword List: Architecture, Integration, CaaS, Docker, Kubernetes, platform, API, environments, development, production Licensing information: This work is licensed under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0 DEED https://creativecommons.org/licenses/by-sa/4.0/) Disclaimer Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union. The European Union cannot be held responsible for them. D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 3 of 46 www.emerald-he.eu Document Description Version Date Modifications Introduced Modification Reason Modified by v0.1 11.02.2025 ToC defined TECNALIA v0.2 25.02.2025 First draft version TECNALIA v0.3 15.03.2025 Updated Sections 1 and 2 TECNALIA v0.4 18.03.2025 Contributions by consortium partners to Section 3 FABA, TECNALIA, Fraunhofer, CNR, SCCH v0.5 03.04.2025 Conclusions and Executive Summary. Sent to QA review TECNALIA v0.6 14.04.2025 Addressed recommendations from QA review. Sent to final review. Fraunhofer, TECNALIA v0.7 19.04.2025 Addressed recommendations from final review TECNALIA v1.0 30.04.2025 Final version submitted to the European Commission TECNALIA D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 4 of 46 www.emerald-he.eu Table of contents Terms and Abbreviations .............................................................................................................. 7 Executive Summary ....................................................................................................................... 8 1 Introduction ........................................................................................................................... 9 1.1 About this deliverable .................................................................................................... 9 1.2 Document structure ....................................................................................................... 9 2 Integration Overview ........................................................................................................... 10 2.1 Architecture Overview ................................................................................................. 10 2.1.1 Workflows.......................................................................................................... 11 2.1.1 Design of the CI/CD Solution ............................................................................. 13 2.2 Components Integrated in the EMERALD Framework v1 ............................................ 13 2.3 Test Bed Environment ................................................................................................. 14 2.3.1 Container orchestration .................................................................................... 16 2.3.2 Storage ............................................................................................................... 17 2.3.3 Docker registry .................................................................................................. 17 2.3.4 Network ............................................................................................................. 18 2.3.5 Dashboard ......................................................................................................... 18 2.3.6 Certificates ......................................................................................................... 19 2.3.7 Deployment view ............................................................................................... 19 2.4 Steps to Integrate a Component ................................................................................. 20 2.5 Overall status of the integration .................................................................................. 21 3 Integration of Components ................................................................................................. 25 3.1 Evidence Collectors ...................................................................................................... 25 3.1.1 AI-SEC ................................................................................................................. 25 3.1.2 AMOE ................................................................................................................. 26 3.1.3 Clouditor-Discovery ........................................................................................... 28 3.1.4 Codyze ............................................................................................................... 28 3.1.5 eknows-e3 ......................................................................................................... 30 3.2 Evidence Assessment and Certification ....................................................................... 31 3.2.1 TWS .................................................................................................................... 31 3.2.2 MARI .................................................................................................................. 34 3.2.3 RCM ................................................................................................................... 34 3.2.4 Orchestrator ...................................................................................................... 37 3.2.5 Evidence Store ................................................................................................... 40 3.2.6 Assessment ........................................................................................................ 41 3.2.7 Evaluation .......................................................................................................... 42 3.3 EMERALD UI ................................................................................................................. 43 4 Conclusions .......................................................................................................................... 45 5 References ........................................................................................................................... 46 List of tables TABLE 1. COMPONENTS IN THE EMERALD FRAMEWORK V1 .................................................................. 13 D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 5 of 46 www.emerald-he.eu TABLE 2. INTEGRATION STATUS .......................................................................................................... 21 TABLE 3. POINT-TO-POINT INTEGRATION STATUS .................................................................................. 23 TABLE 4. INTEGRATION STATUS OF AI-SEC WITH OTHER EMERALD COMPONENTS .................................... 26 TABLE 5. INTEGRATION STATUS OF AMOE WITH OTHER EMERALD COMPONENTS .................................... 28 TABLE 6. INTEGRATION STATUS OF CLOUDITOR-DISCOVERY WITH OTHER EMERALD COMPONENTS ............. 28 TABLE 7. INTEGRATION STATUS OF CODYZE WITH OTHER EMERALD COMPONENTS ................................... 29 TABLE 8. INTEGRATION STATUS OF EKOWS-E3 WITH OTHER EMERALD COMPONENTS ............................... 31 TABLE 9. INTEGRATION STATUS OF TWS WITH OTHER EMERALD COMPONENTS ....................................... 33 TABLE 10. INTEGRATION STATUS OF MARI WITH OTHER EMERALD COMPONENTS ................................... 34 TABLE 11. INTEGRATION STATUS OF THE RCM WITH OTHER EMERALD COMPONENTS .............................. 37 TABLE 12. INTEGRATION STATUS OF ORCHESTRATOR WITH OTHER EMERALD COMPONENTS ...................... 40 TABLE 13. INTEGRATION STATUS OF EVIDENCE STORE WITH OTHER EMERALD COMPONENTS ..................... 41 TABLE 14. INTEGRATION STATUS OF ASSESSMENT WITH OTHER EMERALD COMPONENTS .......................... 42 TABLE 15. INTEGRATION STATUS OF EVALUATION WITH OTHER EMERALD COMPONENTS .......................... 43 TABLE 16. INTEGRATION STATUS OF EMERALD UI WITH OTHER EMERALD COMPONENTS ........................ 44 List of figures FIGURE 1. EMERALD COMPONENTS .................................................................................................. 10 FIGURE 2. PARTICIPATION OF THE COMPONENTS IN THE EMERALD BLUEPRINT FOR AUDIT PREPARATION ...... 12 FIGURE 3. MERGE REQUEST AND GENERIC CI/CD PIPELINES ................................................................... 13 FIGURE 4. INTEGRATION AND PRODUCTION ENVIRONMENTS IN THE EMERALD CAAS FRAMEWORK ............. 14 FIGURE 5. URL NAMING CONVENTION FOR INTEGRATION/PRODUCTION ENVIRONMENTS ............................ 15 FIGURE 6. KUBERNETES CLUSTER INSTALLATION WITH RKE2 ................................................................... 16 FIGURE 7. LONGHORN DASHBOARD IN RANCHER................................................................................... 17 FIGURE 8. EMERALD DOCKER REGISTRY ............................................................................................. 18 FIGURE 9. RANCHER DASHBOARD....................................................................................................... 19 FIGURE 10. DEPLOYMENT DIAGRAM ................................................................................................... 20 FIGURE 11. COMPONENT INTEGRATION MAIN STEPS.............................................................................. 21 FIGURE 12. EVIDENCE COLLECTORS IN THE EMERALD ARCHITECTURE ..................................................... 25 FIGURE 13. ASSESSMENT TOOLS IN THE EMERALD ARCHITECTURE ......................................................... 31 FIGURE 14. EMERALD UI IN THE ARCHITECTURE ................................................................................. 43 List of listings LISTING 1. AMOE API OVERVIEW ...................................................................................................... 27 LISTING 2. TWS API ENDPOINTS FOR ACCOUNT MANAGEMENT ............................................................. 32 LISTING 3. TWS API ENDPOINTS FOR USERS MANAGEMENT .................................................................. 32 LISTING 4. TWS API ENDPOINTS FOR INFORMATION REGISTRATION ........................................................ 33 LISTING 5. TWS API ENDPOINTS FOR INFORMATION ACCESS .................................................................. 33 LISTING 6. TWS API ENDPOINTS FOR INTEGRITY VERIFICATION ............................................................... 33 LISTING 7. MARI API ENDPOINTS FOR MAPPING .................................................................................. 34 LISTING 8. RCM API ENDPOINTS FOR SCHEMA INFORMATION ................................................................ 35 LISTING 9. RCM API ENDPOINTS FOR CONTROL INFORMATION ............................................................... 36 LISTING 10. RCM API ENDPOINTS FOR METRIC INFORMATION ............................................................... 36 LISTING 11. RCM API ENDPOINTS FOR SIMILAR CONTROL RESOURCE ....................................................... 36 LISTING 12. RCM API ENDPOINTS FOR QUESTIONNAIRE RESOURCES ....................................................... 37 LISTING 13. ORCHESTRATOR API ENDPOINTS FOR ASSESSMENT RESULTS ................................................... 38 LISTING 14. ORCHESTRATOR API ENDPOINTS FOR METRICS .................................................................... 38 LISTING 15. ORCHESTRATOR API ENDPOINTS FOR TARGETS OF EVALUATION .............................................. 39 D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 6 of 46 www.emerald-he.eu LISTING 16. ORCHESTRATOR API ENDPOINTS FOR HANDLING CERTIFICATES .............................................. 39 LISTING 17. ORCHESTRATOR API ENDPOINTS FOR CERTIFICATES (PUBLICLY AVAILABLE) ............................... 39 LISTING 18. ORCHESTRATOR API ENDPOINTS FOR CATALOGUES .............................................................. 40 LISTING 19. ORCHESTRATOR API ENDPOINTS FOR AUDIT SCOPES ............................................................. 40 LISTING 20. EVIDENCE STORE API ENDPOINTS ...................................................................................... 41 LISTING 21. ASSESSMENT API ENDPOINT FOR EVIDENCE ......................................................................... 42 LISTING 22. EVALUATION API ENDPOINTS............................................................................................ 42 D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 7 of 46 www.emerald-he.eu Terms and Abbreviations AI Artificial Intelligence AI-SEC AI Security Evidence Collector AIC4 AI Cloud Service Compliance Criteria Catalogue AMOE Assessment and Management of Organizational Evidence API Application Programming Interface AWS Amazon Web Services BSI Bundesamt für Sicherheit in der Informationstechnik CaaS Compliance-as-a-Service1 CI/CD Continuous Integration / Continuous Delivery CLI Command Line Interface CM Compliance Manager CSP Cloud Service Provider EC European Commission EUCS European Cybersecurity Certification Scheme for Cloud Services GA Grant Agreement to the project GB GigaByte gRPC Google Remote Procedure Call HTTP Hypertext Transfer Protocol HTTPS Hypertext Transfer Protocol Secure IAM Identity and Access Management IaC Infrastructure as Code IP Internet Protocol JSON JavaScript Object Notation KR Key Result MARI Mapping Assistant for Regulations with Intelligence ML Machine Learning MS Milestone NLP Natural Language Processing OSCAL Open Security Controls Assessment Language OS Operating System RAM Random Access Memory RBAC Role-Based Access Control RCM Repository of Controls and Metrics REST Representational State Transfer RKE Rancher Kubernetes Engine SSL Secure Sockets Layer TWS Trustworthiness System UI/UX User Interface / User Experience URL Uniform Resource Locator VCS Version Control System VM Virtual Machine WP Work Package 1 Please note that in previous deliverables and in the DoA, the term Certification-as-a-Service was used to stand for CaaS. Compliance has now been introduced to clarify that EMERALD can be used to assess both normative models and internal organizational models. D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 8 of 46 www.emerald-he.eu Executive Summary The deliverable D1.7 presents the initial integrated solution of the EMERALD framework, which includes the integration of the components developed in the project's technical work packages. This document accompanies the software deliverable and provides an overview of the integration approach, the status of the integration, and the components involved. This deliverable is related to Work Package 1 (WP1), which focuses on the project's concept and methodology. The integrated solution is a crucial part of the project as it enables collaboration and communication between the different components developed in other work packages. The integration approach is described first. The integration of the components is based on two main pipelines that automate it: the first one builds the project, creating the Docker images and pushing them to the Artifactory. The second one deploys the components to the test bed environment and verifies it. The test bed is composed by two environments: integration and production and is based on OpenStack Virtual Machines. A three-node Kubernetes cluster is mounted on top of them. An eight-step procedure is defined for the integration of a component in the EMERALD framework. The status of the integration of each component is provided, including the APIs published, and the interconnection with the rest of components. The main results of this deliverable include the development of an initial prototype of the EMERALD framework; the implementation of an integration strategy that allows collaboration between components; the deployment of components in the integration and production environments; and the documentation of the integration status of each component, providing a basis for future improvements and developments. Future related work in the project will focus on the continuous integration of the EMERALD framework, including new releases with more functionalities and feedback from the users of the first version of the framework. This work will be reflected in a second version of the deliverable, D1.8, scheduled for month 30 of the project, which will include the updated status of the integration of the EMERALD components. D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 9 of 46 www.emerald-he.eu 1 Introduction 1.1 About this deliverable This is the companion document of the software deliverable D1.7, which aims to have an initial prototype of the EMERALD Compliance-as-a-Service 2 (CaaS) Framework that integrates the components developed by the other technical work packages. This first version of the integrated solution corresponds to the Milestone MS3 – Integrated Audit Suite v1 and is mainly based on the version v1 of the EMERALD components (M12), although some additional development made until M15 has also been included in some cases. All the referred software is available in the project’s public Gitlab (https://git.code.tecnalia.dev/emerald/public). The document includes first an overview of the integration approach, to provide the reader an overview of what components are integrated, where and how, and the status of the overall integration task. It also describes the hardware equipment used to setup the test bed 3 , the resources needed for the installation, and the configuration. The methodology through which a component is integrated in the framework is introduced as well. The document also includes the description of the main workflow in several scenarios and briefly describes the CI/CD solution that has been implemented to support the development and integration activities of the EMERALD framework. Finally, the document provides a detailed overview of the current status of the integration of all components of the EMERALD framework. A second version of the deliverable is planned for month 30 of the project. This second version will incorporate advancements and improvements made in the time between the two releases. It is expected that some of the improvements will come from user feedback, testing, and discussions on the current version. 1.2 Document structure The remainder of the document is organized as follows: Section 2 presents a general description of the integration strategy and tools. It gives an overview of the EMERALD CaaS framework, the resources used for the test bed environments, the integration steps for each component, and the CI/CD implementation supporting the integration of the EMERALD framework. An overall integration status is also provided. Section 3 provides in more detail the integration status of each EMERALD component, in terms of the connection with other components. Section 4 presents the conclusions, including a summary of the main outcomes of the deliverable. 2 Please note that in previous deliverables and in the DoA, the term Certification-as-a-Service was used to stand for CaaS. Compliance has now been introduced to clarify that EMERALD can be used to assess both normative models and internal organizational models. 3 A “Test Bed” refers to the setup where the testing activities take place. It includes the combination of hardware, software, network configurations, and other necessary components that provide the infrastructure which aims to simulate the real-world conditions under which the software will operate. (see more at https://testingfundamental.com/test-bed) D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 16 of 46 www.emerald-he.eu We have followed an Infrastructure as Code (IaC) approach for the deployment. For the creation and configuration of the cluster we have used OpenTofu 13 and Ansible 14 technologies. OpenTofu is used to create the nodes, networks, network interfaces, and security groups among other infrastructural elements. Ansible is used to configure the nodes with the software packages required to implement the Kubernetes cluster. The IaC files are also under a configuration management process, in the Gitlab repository of the project. The usage of IaC provides several advantages to the project management: • Allows the redeployment of the cluster from scratch, if we need to migrate. • Simplifies the horizontal scalation of the Kubernetes , if more capacity is required. • Reusable by pilots, in case they have similar infrastructure. 2.3.1 Container orchestration The EMERALD framework functionalities are made up by the collaboration of micro-services, which communicate each other through APIs, are packaged in docker images and run in containers. Kubernetes orchestrates all these containers in a virtual environment running in a highly available cluster. We also use an IaC approach based on Kustomize 15 to describe the deployment and collaboration of all components of the EMERALD project. The container orchestration is stored in a separate Gitlab repository of the project named “CaaS Framework”. The repository contains a folder with the details of the deployment of the individual components (called components), and other folders that describe the environments: integration and production. Figure 6. Kubernetes cluster installation with RKE2 13 https://opentofu.org 14 https://www.ansible.com 15 Kustomize traverses a Kubernetes manifest to add, remove or update configuration options without forking. More information is available at https://kustomize.io/ D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 17 of 46 www.emerald-he.eu 2.3.2 Storage The micro-services can store their data in an easy and secure way thanks to the configuration of a distributed filesystem provided by Longhorn 16 . Indeed, each node of the cluster provides 200 GB of storage, managed by Longhorn and is exposed as a single, unified cluster filesystem. Thus, the data is replicated across the three nodes, and a total of 989 GB fault-tolerant and high availability of storage are assured, as shown in Figure 7. Figure 7. Longhorn dashboard in Rancher 2.3.3 Docker registry The micro-services running on the Kubernetes cluster are packaged in Docker images and stored in a private Docker Registry running in the TECNALIA infrastructure’s Artifactory 17 . To access the Docker Registry, a Kubernetes secret has been created with the credentials. This allows Kubernetes to pull the micro-service images and then run them on the cluster. The images are pushed to the Docker registry by the GitLab CI/CD pipelines in the following URL according to the structure agreed for the project, as shown in Figure 8. artifact.tecnalia.dev/ui/native/emerald-docker-dev-local/<component>/ 16 Longhorn is an open-source, cloud-native distributed storage solution for delivering block storage persistent with low requirements and overhead. For more details see https://rook.io/docs/rook/v1.8/ 17 https://jfrog.com/artifactory/ D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 18 of 46 www.emerald-he.eu Figure 8. EMERALD Docker registry 2.3.4 Network On the Kubernetes cluster, a nginx 18 service is configured as a proxy to redirect all the requests to the correct micro-service component. The binding between the nginx service and the public IP is setup with KubeVip 19 , a network load-balancer that associates the public IP to the nginx service and uses standard routing protocols to make available (part of) the network behind the Kubernetes cluster. It is essential for the EMERALD cluster because, unlike a public cloud provider cluster, nginx has no load balancer, and Kubernetes does not provide it by itself. 2.3.5 Dashboard We have two accounts in Kubernetes, with different permissions: one that has access to all cluster resources and to the administration options (“admin”), and one that has the permissions restricted to the integration and production namespaces (“emerald_developer”). Rancher provides a web-based dashboard for the Kubernetes cluster (see Figure 9). It is helpful to deploy containerised applications to a Kubernetes cluster, troubleshoot them, and manage cluster resources. 18 https://www.nginx.com/ 19 https://kube-vip.io/ D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 19 of 46 www.emerald-he.eu Figure 9. Rancher Dashboard 2.3.6 Certificates Access to the Dashboard is secure via HTTPS. The certificates are installed using Cert-Manager 20 . Cert-Manager automates the provisioning of certificates and provides a set of custom resources to issue certificates and attach them to services. EMERALD secures web apps and APIs with SSL certificates from Let’s Encrypt 21 . We installed Cert-Manager using the manifest file, created an issuer that uses the Let’s Encrypt API for the Dashboard domain and exposed it over HTTPS. The Dashboard is exposed over HTTPS at the address: https://k8so.emerald.digital.tecnalia.dev/dashboard/. 2.3.7 Deployment view The EMERALD CaaS Framework is deployed on the Kubernetes cluster. As we explained in Section 2.3, the cluster is currently composed of three nodes. Figure 10 shows a deployment diagram of the solution, showing all components deployed (in blue). Each component is composed by one or more containers, represented by artifacts (white boxes). 20 https://cert-manager.io/docs/ 21 https://letsencrypt.org/ D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 20 of 46 www.emerald-he.eu Figure 10. Deployment diagram Figure 10 represents the deployment at a given time. The distribution of the artifacts among the nodes is managed by Kubernetes, and it is possible that a single component has its artifacts distributed on different nodes (e.g., AMOE or RCM). This distribution is automatically modified by Kubernetes attending to its own performance and resources management criteria. The Codyze, eknows-e3 and AI-SEC components are not present in Figure 10, as they are evidence extractors that are intended to run in a separate environment, alongside a GitLab runner that gives them access to the files to be analysed. 2.4 Steps to Integrate a Component Once the Test Bed environment has been installed and properly configured, the next step is the deployment of all components in the cluster. To better organize the integration, we have adopted the following methodology, which presents the actions to be taken until the complete release of the EMERALD Framework. Figure 11 shows the main steps in the integration and deployment of a component 22 : 1. The source code of each component must be uploaded to the private GitLab repository. 2. Once finalised and tested, each component must be containerised into a Docker image, so it must provide dockerfile(s) in the GitLab repository, which help automate the building of the images after any changes in the code. 3. The Docker image must be made available on the private docker registry Artifactory. 4. The configuration and side services for each component should be specified in the CaaS Framework repository. 5. The configuration must be manually tested to perform standalone, point-to-point, and workflow tests, to verify that each component is deployed correctly, communicates with its peers, and the workflows (described in section 2.1.1) are correctly implemented. 6. If the tests are passed, the release can be merged with the integration environment. 7. Automated integration tests are performed in the integration environment. 8. If the tests are passed, the version is promoted to the production environment and a new release is created. 22 The integration of non-open source component skips steps 1 and 2. D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 21 of 46 www.emerald-he.eu Figure 11. Component integration main steps The integration plan includes three phases that will be completed in months M18, M30, and M34, respectively. Currently, we have performed the first round, where the integration of components has been carried out manually by each partner. During this first round, WP1 delivered a workshop to the rest of the consortium to introduce the main concepts of Docker and Kubernetes and explained the integration steps. All components were developed concurrently, and their code history was tracked using the GitLab version control system. Besides, a semantic release numbering was implemented to track the progress of the project. All components are containerised and have been deployed on the project-internal integration server. Dockerfile recipes are available to easy recreate the integration environment. All REST API endpoints are exposed on a common network to enable communication between components. The EMERALD Framework is installed entirely using Kubernetes manifests. In phases 2 and 3 we expect to have the deployment fully automated, governed by the CI/CD pipelines. We will also set-up the production environment, with a stable version always available in it. And finally, the installation will be implemented in the pilots. 2.5 Overall status of the integration This section provides an overview of the integration status of the components in the EMERALD framework. Table 2 shows the steps to be carried out for the integration of each component, as well as their degree of completion. Section 3 provides more details on the level of integration of each component. Table 2. Integration status Component License Gitlab Repo Public Repo README Docker Images OpenAPI spec K8s file Deploy pipeline Deployed (integr.) Integration URL AMOE √ (Apache) √ √ √ √ √ √ √ √ URL MARI √ (Apache) √ √ √ √ √ √ √ √ URL RCM √ (Apache) √ √ √ √ √ √ √ √ URL TWS * √ (Propietary) N/A N/A √ √ √ √ √ √ URL Assessment √ (Apache) √ √ √ √ √ √ √ √ URL Clouditor-Discovery √ (Apache) √ √ √ √ N/A √√√URL Evaluation √ (Apache) √ √ √ √ √ √ √ √ URL Evidence Store √ (Apache) √ √ √ √ √ √ √ √ URL Orchestrator √ (Apache) √ √ √ √ √ √ √ √ URL Codyze √ (Apache) √ √ √ X√ (CLI) N/A X X N/A eKnows-e3 * √ (Apache, others) √ √ & N/A √ √ √ (CLI) N/A X X N/A AI-SEC √ (Apache) √ √ √ X X N/A X X N/A Emerald UI √ (Apache) √ √ √ √ √ √ √ √ URL D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 22 of 46 www.emerald-he.eu The starting point of the integration process is the source code of the components, which has been uploaded to the public GitLab repository 23 (except for two of them -TWS and eknows-e3– which are not open source licensed). The repository of each component contains a dockerfile. The respective images created have been uploaded to the Docker repository in Artifactory. The next step is to deploy the images on the Kubernetes cluster. For this, several manifest files have been developed for each component, depending on their nature (pods, services, volumes, etc.). In the early stages of the development, the integration process allowed a manual deployment using kustomize and kubectl 24 . This allows for the identification of bugs/adjustments in an agile way, without waiting for an automated deployment process to be completed. This process has been further automated through the GitLab CI pipelines, which are detailed in D1.6 [4]. The last column of Table 2 indicates if the component is available in the integration environment. All of them are deployed, excepting those that are not to be integrated in the CaaS Framework but in the pipeline to analyse files (i.e., Codyze, eknows-e3, and AI-SEC). The final and fundamental part of the integration has to do with the communication among components, which is done through the APIs defined and developed in EMERALD. Besides the details provided in the third section for each component, Table 3 presents a consolidated view of the current status of the interaction of each component with the others. The status has been categorized into several stages 25 , reflecting the progress made in integrating the component into the EMERALD Framework: • Not Started: The integration process has not yet begun. • Developing API: The component is currently in the process of developing its API. This stage involves defining how the component will interact and its implementation. • API Finished: The API development has been completed and is ready for testing. • Tested Locally: The component has undergone local testing, verifying its functionality in isolation. While it works as intended on its own, it has not yet been tested in conjunction with other components. • Connected: The component has successfully established connections with other components. Data exchange can occur, but further testing is needed to ensure full compatibility. • Testing: The integration of the component with others is currently being tested. This phase involves checking the data flow between the components to identify and fix any issues that may arise. • Integrated: The component has been fully integrated into the framework. It has passed the necessary tests and is functioning as intended, interacting seamlessly with other components in the EMERALD system. Please note that in Table 3, the column “Component A” refers to the component that implements the API and “Component B” is the component that invokes it. 23 https://git.code.tecnalia.dev/emerald/public 24 A command line tool for communicating with a Kubernetes cluster's control plane, using the Kubernetes API. See https://kubernetes.io/docs/reference/kubectl/ for details. 25 This categorization was first used in deliverable D3.5 [16] D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 23 of 46 www.emerald-he.eu Table 3. Point-to-point integration status Component A Component B Status Comment AI-SEC EMERALD UI Not started AI-SEC is currently in the tool testing phase and has not yet started its integration AMOE Evidence Store Testing Waiting for Ontology and full integration of new metrics based on updated data model AMOE RCM Tested locally / Connected Tested with initial API. Waiting for adjustment to new metric data model and inclusion of extended set of metrics AMOE Orchestrator Not started Waiting for updates to the Orchestrator API AMOE EMERALD UI Testing, Connected Collected files and extracted results. Full implementation remains to be tested. Also, extensive testing in EMERALD integration setup remains to be done. The Keycloak configuration needs to be updated for a stable deployment and test setup. Clouditor-Discovery Evidence Store Connected Testing pending Codyze [CI/CD] * Tested locally Codyze is integrated as a CI/CD component. Currently, a proof-ofconcept integration for CodyzeProvenance exists for GitLab. For Codyze-Compliance, a similar integration is planned. Codyze Evidence Store Tested locally We can send pieces of evidence. However, they are not fully filled. Codyze TWS Not started Currently postponed in favour of the integration with Evidence Store and awaiting final API specification of TWS. eknows-e3 [CI/CD] * Tested locally The CI/CD component uses eknows-e3 to extract and save evidence in the Evidence Store. A demo showcases how the eknows-e3 component can be integrated. Integration tests are currently being developed. eknows-e3 Evidence Store Testing Communication is implemented, integration tests are in development. TWS EMERALD UI Developing API Currently updating the API details due to the migration process. TWS Evidence Store Developing API Currently updating the API details due to the migration process. TWS Evidence collectors Developing API Currently updating the API details due to the migration process. MARI RCM Developing API New API defined . RCM EMERALD UI Developing API Updating / extending the API. RCM ClouditorOrchestrator Developing API Changes are needed due to data model updates. RCM MARI Developing API New mapping API already defined. D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 24 of 46 www.emerald-he.eu Component A Component B Status Comment RCM AMOE Connected Changes are needed due to data model updates. Orchestrator Assessment API Finished - Orchestrator EMERALD UI API finished Requires coordination with WP4 and testing. Orchestrator RCM Developing API - Orchestrator Assessment Connected Testing pending. Orchestrator Evaluation Connected Testing pending. Evidence Store Assessment Connected Testing pending. Evidence Store Orchestrator Connected Testing pending. Evidence Store AMOE Testing - Evidence Store Codyze Tested locally - Evidence Store eknows-e3 Testing - Evidence Store AI-SEC Testing - Evidence Store Clouditor-Discovery Connected Testing pending. Assessment Evidence Store Connected Testing pending. Assessment Orchestrator Connected Testing pending. Assessment TWS Developing API To be tested. Evaluation Orchestrator Connected Testing pending. EMERALD UI AMOE Connected Testing remains to be done. Only a subset of the endpoints has been fully integrated yet. EMERALD UI Orchestrator Developing API Waiting for Orchestrator results. EMERALD UI RCM Tested locally Waiting for updates to the API. EMERALD UI TWS Not started Discussion for endpoints and integration needed. The information reflected in Table 3 can be taken as a basis for a consolidated status of the point-to-point integration. If “Not started” is considered as 0% progress, and “Integrated” is considered as 100% progress (with “Tested locally” being 50%), an approximation to the global value can be calculated, resulting in 40% at the end of this point-to-point integration at M18. The majority of the 38 elements in the table are in the “Connected” status (13), followed by the “Developing API” status (11). D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 25 of 46 www.emerald-he.eu 3 Integration of Components This section provides more details on the integration status of each EMERALD component. The components are grouped into three groups: (i) Evidence Collectors; (ii) Evidence Assessment and Certification; and (iii) User Interface. Each component is introduced by a short description, followed by the expected behaviour concerning inputs and outputs. Next, the APIs published by the component, or the Command Line Interfaces (CLI), if applicable, are listed. Finally, a status of the integration with other components is provided. 3.1 Evidence Collectors Evidence collectors –highlighted in the Figure 12 below– are the components in charge of collecting different forms of data from the targets of evaluation and providing them as evidence that is then processed in the EMERALD framework to decide on compliance. Figure 12. Evidence collectors in the EMERALD architecture 3.1.1 AI-SEC AI-SEC is an evidence collector designed to extract relevant information from machine learning models. Based on the Criteria Catalogue for AI Cloud Services (AIC4) [11], AI-SEC extracts various characteristics of machine learning models, e.g. robustness, privacy levels and explainability. AISEC establishes a process that contains methods for extracting these features. These methods are typically applicable to both image and language models. 3.1.1.1 Expected behaviour (inputs/outputs) The expected input should be the machine learning model and its (partial) training data, while the output will be the computed evidence. AI-SEC is connected to the EMERALD UI and the Evidence Store: • EMERALD UI: It connects to AI-SEC for uploading, downloading, viewing, and deleting policy documents. AI-SEC will be controlled by the user via the EMERALD UI, which connects to the API. • Evidence Store: Evidence will be forwarded to the Evidence Store. D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 32 of 46 www.emerald-he.eu • Assessment: The interaction with this component occurs in two ways: o The Assessment component provides information (proofs of integrity) related to evidence and assessment results to be recorded on the Blockchain. o The automatic verification service requests the current values of evidence and assessment results stored in EMERALD’s internal evidence storage to validate their integrity against the information previously recorded on the Blockchain. • Evidence collectors: Proofs of integrity for evidence can be directly provided from the evidence collectors. In particular, Codyze will be considered as a proof of concept. • EMERALD UI: The graphical interface of the TWS automatic verification service is integrated into the EMERALD UI, allowing auditors to easily verify the trustworthiness of evidence and assessment results, and determine their reliability. 3.2.1.2 Published APIs The API endpoints of the TWS are listed below. Listing 2. TWS API Endpoints for Account Management Listing 3. TWS API Endpoints for Users Management D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 33 of 46 www.emerald-he.eu Listing 4. TWS API Endpoints for Information Registration Listing 5. TWS API Endpoints for Information Access Listing 6. TWS API Endpoints for Integrity Verification 3.2.1.3 Integration Status Currently, the TWS has been successfully deployed on the Kubernetes cluster. However, it has been recently migrated from Quorum to the Alastria Blockchain network. This migration has slightly delayed the integration process with other EMERALD components. Table 9 provides the current state of the connections of the TWS with other components. Table 9. Integration status of TWS with other EMERALD components Component Status Comment EMERALD UI Developing API Currently updating the API details due to the migration process. Evidence Store Developing API Evidence collectors (Codyze) Developing API D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 34 of 46 www.emerald-he.eu 3.2.2 MARI The Mapping Assistant for Regulations with Intelligence (MARI) is an intelligent system for compliance management. As described in D3.3 [9], its main functionality is to automatically associate relevant metrics with controls and facilitate the mapping of controls across multiple certification schemes. This automation significantly reduces manual effort and improves performance in compliance management processes. The MARI is built as an NLP-based tool, leveraging a sentence transformer model to generate vector embeddings that capture the semantic meaning of controls and metrics. The associations between controls and metrics, as well as between controls across different certification schemes, are then performed by measuring the similarity between these embeddings in the vector space. 3.2.2.1 Expected behaviour (inputs/outputs) The MARI exchanges information exclusively with the RCM. • RCM: It sends the mapping requests to the MARI, including information about the certification schemes and the metrics. Once the MARI performs the mappings, the results are sent back to the RCM, which receives and stores them for further use. 3.2.2.2 Published APIs The MARI provides two endpoints for mapping controls and metrics: the mapControls endpoint that maps controls from a schema to another by evaluating a similarity threshold, actively matching controls that meet the required standard; and the mapMetrics2Controls endpoint that links metrics to the corresponding controls based on the same similarity principle. Listing 7. MARI API Endpoints for mapping 3.2.2.3 Integration Status The MARI has already been deployed on the Kubernetes cluster. MARI interacts exclusively with the RCM through a predefined API. Table 10 provides the current state of connections of the MARI with the RCM. Table 10. Integration status of MARI with other EMERALD components Component Status Comment RCM Developing API New API defined 3.2.3 RCM The Repository of Control and Metrics (RCM) is a smart catalogue of controls and metrics. As described in D3.3 [9], the RCM supports multi-scheme and multi-level compliance and incorporates the definition of the metrics used in EMERALD to obtain and assess evidence. The RCM also provides mechanisms to update the catalogues and allow OSCAL-based [13] import/export to facilitate the reuse and composition of the catalogue elements; stores the D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 35 of 46 www.emerald-he.eu mapping of controls and metrics provided by the MARI component; and includes a selfassessment questionnaire to assess EUCS [14] compliance. 3.2.3.1 Expected behaviour (inputs/outputs) The RCM sends and receives information from different sources. • Clouditor-Orchestrator: It retrieves information about schemes and metrics from the RCM, which is then used to configure extractors and organize evidence. • MARI: It receives the mapping requests, which include information about schemes and metrics, from the RCM. The responses (mappings) are then sent back to the RCM, which stores them for further use. • EMERALD UI: When the user navigates through the content of the repository, the EMERALD UI calls the RCM API. The information required is packed in JSON format in the REST call and sent to the EMERALD UI for displaying. The same happens, when the user creates new security schemes or fill in the self-assessment questionnaire. • AMOE: It receives the definition of the security metrics that are used to extract and evaluate evidence from policy documents from the RCM. 3.2.3.2 Published APIs The API endpoints of the RCM component are listed below. Listing 8. RCM API Endpoints for schema information D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 36 of 46 www.emerald-he.eu Listing 9. RCM API Endpoints for control information Listing 10. RCM API Endpoints for Metric information Listing 11. RCM API Endpoints for similar control resource D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 37 of 46 www.emerald-he.eu Listing 12. RCM API Endpoints for Questionnaire resources 3.2.3.3 Integration Status The RCM has already been deployed on the Kubernetes cluster. Table 11 provides the current state of connections of the RCM with other components. Table 11. Integration Status of the RCM with other EMERALD components Component Status Comment EMERALD UI Developing API Updating / extending the API Orchestrator Developing API - MARI Developing API New API already defined AMOE Connected Changes needed 3.2.4 Orchestrator The Orchestrator is the central orchestration point in the EMERALD framework. As described in D3.3 [9], the Orchestrator serves as a key element that manages the compliance process within the EMERALD framework, linking various components together. This component is also responsible for making the final compliance decision, assessing whether a target of evaluation adheres to a specified security standard. 3.2.4.1 Expected behaviour (inputs/outputs) The Orchestrator sends and receives information from different sources. • EMERALD UI: It receives information from the Orchestrator to be displayed to the user, e.g. audit scope or evidence. • RCM: It provides the Orchestrator with catalogue and metric information. • Assessment: Assessment results, as well as evidence, are sent by the Assessment to the Orchestrator. • Evaluation: For evaluating the compliance of controls of a security catalogue, the Orchestrator sends assessment results to the Evaluation and gets the compliance status of each control (i.e. the evaluation result). • Evidence Store: The Orchestrator pulls evidence from it. D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 38 of 46 www.emerald-he.eu 3.2.4.2 Published APIs The API endpoints of the Orchestrator for handling assessment results and tools are listed below. Listing 13. Orchestrator API endpoints for assessment results Listing 14. Orchestrator API endpoints for metrics D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 39 of 46 www.emerald-he.eu Listing 15. Orchestrator API endpoints for targets of evaluation Listing 16. Orchestrator API endpoints for handling certificates Listing 17. Orchestrator API endpoints for certificates (publicly available) D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 40 of 46 www.emerald-he.eu Listing 18. Orchestrator API endpoints for catalogues Listing 19. Orchestrator API endpoints for audit scopes 3.2.4.3 Integration Status The Orchestrator has already been deployed on the Kubernetes cluster. Table 12 provides the current state of connections of the Orchestrator with other components. Table 12. Integration status of Orchestrator with other EMERALD components Component Status Comment EMERALD UI API finished Requires coordination with WP4 and testing. RCM Developing API - Assessment Connected - Evaluation Connected - Evidence Store Connected - 3.2.5 Evidence Store The Evidence Store serves as a central repository for evidence collected from various evidence collectors. It retrieves evidence from the evidence collectors, saves them in a Postgres database, and forwards evidence to the Assessment and the TWS to improve the integrity of the evidence. A detailed description can be found in the deliverables D3.3 [9] . D1.7 - EMERALD Integrated solution - v1 Version 1.0 – Final. Date: 30.04.2025 © EMERALD Consortium Contract No. GA 101120688 Page 41 of 46 www.emerald-he.eu 3.2.5.1 Expected behaviour (inputs/outputs) The Evidence Store sends and receives information from different sources: • Assessment: It receives evidence from the Evidence Store for the assessment. • Orchestrator: It receives evidence from the Evidence Store (and forwards it to the EMERALD UI) • AMOE: It sends evidence to the Evidence Store for storage. • Codyze: It sends evidence to the Evidence Store for storage. • eknows-e3: It sends evidence to the Evidence Store for storage. • AI-SEC: It sends evidence to the Evidence Store for storage. • Clouditor-Discovery: It sends evidence to the Evidence Store for storage. 3.2.5.2 Published APIs The Evidence Store provides the following three endpoints for storing evidence, listing all evidence and getting specific evidence, respectively. Listing 20. Evidence Store API endpoints 3.2.5.3 Integration Status The Evidence Store has already been deployed on the Kubernetes cluster. Table 13 provides the status of the individual connections. The Postgres database is likely to be replaced by a graph database in the future. Table 13. Integration status of Evidence Store with other EMERALD components Component Status Comment Assessment Connected - Orchestrator Connected - AMOE Testing - Codyze Tested locally Simple pieces of evidence are received. eknows-e3 Testing - AI-SEC Testing - Clouditor-Discovery Connected - 3.2.6 Assessment As described in D3.3 [9], the Assessment is tasked with assessing evidence according to specific metrics established within the EMERALD framework. 3.2.6.1 Expected behaviour (inputs/outputs) The Assessment sends and receives information from different sources: