D1.3 EMERALD solution architecture - v1

© EMERALD Consortium Contract No. GA 101120688 Page 1 of 73 Deliverable D1.3 EMERALD solution architecture-v1 Editor(s): Iñaki Etxaniz Responsible Partner: TECNALIA Research & Innovation Status-Version: Final-v1.0 Date: 31.10.2024 Type: R Distribution level (SEN, PU): PU D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 2 of 73 Project Number: 101120688 Project Title: EMERALD Title of Deliverable: D1.3 EMERALD solution architecture-v1 Due Date of Delivery to the EC 31.10.2024 Workpackage responsible for the Deliverable: WP1 - Concept and methodology of EMERALD Editor(s): Iñaki Etxaniz (TECNALIA) Contributor(s): FABA, TECNALIA, Fraunhofer, CNR, SCCH Reviewer(s): Christian Banse (Fraunhofer) Cristina Martínez, Juncal Alonso (TECNALIA) Approved by: All Partners Recommended/mandatory readers: WP1, WP2, WP3, WP4, WP5 Abstract: Initial version of the description and design of the architecture of the EMERALD solution and underlying component integration. Keyword List: Architecture, Requirements, Sequence diagrams, Component cards Licensing information: This work is licensed under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0 DEED https://creativecommons.org/licenses/by-sa/4.0/) Disclaimer Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union. The European Union cannot be held responsible for them. D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 3 of 73 Document Description Version Date Modifications Introduced Modification Reason Modified by v0.1 14.06.2024 Table of Contents, Structure Iñaki Etxaniz (TECNALIA) v0.2 24.09.2024 First draft. Included requirements, Sequence diagrams Iñaki Etxaniz (TECNALIA) v0.3 01.10.2024 Completed context and architecture Iñaki Etxaniz (TECNALIA) v0.4 15.10.2024 Included 3.4 Analysis, Conclusions Ready for internal review Iñaki Etxaniz (TECNALIA) v0.5 24.10.2024 Internal QA Review Christian Banse (Fraunhofer) v0.6 25.10.2024 Addressed comments received in the Internal QA review Iñaki Etxaniz (TECNALIA) v0.7 30.10.2024 Final review Cristina Martínez /Juncal Alonso (TECNALIA) v0.8 31.10.2024 Address comments received in the final review Iñaki Etxaniz (TECNALIA)TECNALIA v1.0 31.10.2024 Submitted to the European Commission Cristina Martínez /Juncal Alonso (TECNALIA) D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 4 of 73 Table of contents Terms and Abbreviations .............................................................................................................. 6 Executive Summary ....................................................................................................................... 7 1 Introduction ........................................................................................................................... 8 1.1 About this deliverable .................................................................................................... 8 1.2 Document structure ....................................................................................................... 8 2 Overview of the EMERALD Framework ................................................................................. 9 2.1 Context diagram ............................................................................................................ 9 2.2 The EMERALD framework ............................................................................................ 11 2.3 Glossary........................................................................................................................ 12 3 EMERALD Framework Requirements .................................................................................. 16 3.1 Methodology and Tools for requirements elicitation ................................................. 16 3.1.1 The process ........................................................................................................ 16 3.1.2 The tools ............................................................................................................ 17 3.2 Functional Requirements ............................................................................................. 20 3.3 Non-Functional Requirements ..................................................................................... 25 3.3.1 Other WP1 requirements .................................................................................. 25 3.3.2 Business driven requirements ........................................................................... 27 3.3.3 UI/UX requirements (usability).......................................................................... 28 3.4 Analysis of Requirements ............................................................................................ 29 3.4.1 Mapping of requirements to KRs ...................................................................... 29 3.4.2 Mapping of requirements to KPIs ...................................................................... 31 3.4.3 Mapping of requirements to Business Driven Requirements ........................... 34 3.4.4 Prioritization and current status........................................................................ 36 3.5 Requirements Summary Dashboard ............................................................................ 37 4 EMERALD Framework detailed view ................................................................................... 40 4.1 Data model .................................................................................................................. 40 4.2 Component description (components cards & sequence diagrams) .......................... 43 4.2.1 Evidence Collectors ........................................................................................... 43 4.2.2 TWS – Trustworthiness System ......................................................................... 52 4.2.3 MARI - Mapping Assistant for Regulations with Intelligence ............................ 55 4.2.4 RCM - Repository of Controls and Metrics ........................................................ 57 4.2.5 Orchestrator ...................................................................................................... 59 4.2.6 Evidence Store ................................................................................................... 62 4.2.7 Assessment ........................................................................................................ 64 4.2.8 Evaluation .......................................................................................................... 66 5 Conclusions .......................................................................................................................... 69 D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 5 of 73 6 References ........................................................................................................................... 70 APPENDIX A: Current status of requirements ............................................................................. 72 List of tables TABLE 1. ROLES IN THE EMERALD ECOSYSTEM .................................................................................... 9 TABLE 2. EMERALD GLOSSARY ......................................................................................................... 12 TABLE 3. REQUIREMENT TEMPLATE .................................................................................................... 17 TABLE 4. COMPONENT CARD TEMPLATE .............................................................................................. 19 TABLE 5. FUNCTIONAL REQUIREMENTS. ............................................................................................... 21 TABLE 6. BUSINESS DRIVEN REQUIREMENTS ......................................................................................... 27 TABLE 7. UI/UX REQUIREMENTS ........................................................................................................ 28 TABLE 8. FUNCTIONAL REQUIREMENTS AND KRS ALIGNMENT MATRIX ...................................................... 29 TABLE 9. FUNCTIONAL REQUIREMENTS AND KPIS ALIGNMENT MATRIX. .................................................... 32 TABLE 10. TECHNICAL REQUIREMENTS VS BUSINESS REQUIREMENTS ALIGNMENT MATRIX........................... 34 TABLE 11. REQUIREMENTS PRIORITIZATION MATRIX .............................................................................. 36 TABLE 12. SUMMARY TABLE OF REQUIREMENTS STATUS AT M12 (BY COMPONENT) ................................... 37 TABLE 13. GENERAL VIEW: COMPONENTS VS PILOT........................................................................... 39 TABLE 14. STATUS OF THE TECHNICAL REQUIREMENTS ........................................................................... 72 List of figures FIGURE 1. EMERALD CONTEXT DIAGRAM ........................................................................................... 10 FIGURE 2. OVERVIEW OF THE EMERALD COMPONENTS........................................................................ 11 FIGURE 3. LIST OF REQUIREMENTS AS ISSUES IN GITLAB (EXCERPT) ........................................................... 19 FIGURE 4. NUMBER OF REQUIREMENTS PER COMPONENT ...................................................................... 38 FIGURE 5. REQUIREMENT STATUS ....................................................................................................... 38 FIGURE 6. REQUIREMENT STATUS PER COMPONENT .............................................................................. 39 FIGURE 7. EMERALD DATA MODEL (D1.1 [1]).................................................................................... 41 FIGURE 7. EMERALD DATA DIAGRAM ................................................................................................ 41 FIGURE 9. AI-SEC SEQUENCE DIAGRAM .............................................................................................. 44 FIGURE 10. AMOE SEQUENCE DIAGRAM ............................................................................................ 46 FIGURE 11. CLOUDITOR-DISCOVERY SEQUENCE DIAGRAM ...................................................................... 48 FIGURE 12. CODYZE SEQUENCE DIAGRAM ............................................................................................ 49 FIGURE 13. OVERVIEW OF EKNOWS PLATFORM COMPONENTS ................................................................ 50 FIGURE 14. EKNOWS SEQUENCE DIAGRAM ........................................................................................... 52 FIGURE 15. TWS SYSTEM RECORDING SEQUENCE DIAGRAM ................................................................... 54 FIGURE 16. TWS SYSTEM VERIFICATION SEQUENCE DIAGRAM ................................................................ 55 FIGURE 17. MARI SEQUENCE DIAGRAM .............................................................................................. 57 FIGURE 18. RCM SEQUENCE DIAGRAM ............................................................................................... 59 FIGURE 19. ORCHESTRATOR SEQUENCE DIAGRAM ................................................................................. 62 FIGURE 20. EVIDENCE STORE SEQUENCE DIAGRAM ................................................................................ 64 FIGURE 21. ASSESSMENT SEQUENCE DIAGRAM ..................................................................................... 66 FIGURE 22. EVALUATION SEQUENCE DIAGRAM ..................................................................................... 68 D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 6 of 73 Terms and Abbreviations AI Artificial Intelligence AI-SEC AI Security Evidence Collector AIC4 AI Cloud Service Compliance Criteria Catalogue AMOE Assessment and Management of Organizational Evidence API Application Programming Interface BDR Business-Driven Requirement CaaS Certification-as-a-Service CI/CD Continuous Integration / Continuous Delivery CKM Cryptography and Key Management CLI Command Line Interface CSA or EU CSA EU Cybersecurity Act CSP Cloud Service Provider CSV Comma-Separated Values CPU Central Processing Unit DoA Description of the Action EBSI European Blockchain Services Infrastructure EC European Commission EUCS European Cybersecurity Certification Scheme for Cloud Services GA Grant Agreement to the project gRPC Google Remote Procedure Call HTTP Hypertext Transfer Protocol ICT Information Communications Technology IEC International Electrotechnical Commission ISO International Organization for Standardization JPA Java Persistence API KPI Key Performance Indicator KR Key Result MARI Mapping Assistant for Regulations with Intelligence ML Machine Learning MS MileStone MVC Model, View, Controller NFR Non-Functional Requirement NLP Natural Language Processing OSCAL Open Security Controls Assessment Language OSS Open-Source Software Protobuf Protocol Buffers RBAC Role-Based Access Control RCM Repository of Controls and Metrics REST Representational State Transfer SARIF Static Analysis Results Interchange Format SDLC Software Development Life Cycle SSI Self-Sovereign Identity System TWS Trustworthiness System UI/UX User Interface / User Experience UML Unified Modelling Language VM Virtual Machine UI/UX User Interface/ User eXperience D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 7 of 73 Executive Summary This deliverable proposes an architecture for the EMERALD framework. It is produced in the context of WP1-Concept and methodology of EMERALD, more concretely in Task 1.2 EMERALD architecture. It provides a general view of the EMERALD framework, which complements the Data Model presented some months before in D1.1 [1]. This document contributes to these outcomes of the work package: • The architecture of the overall EMERALD software suite and the related structural and behavioural models, as well as data modelling and interaction mechanisms definition. • The integration of WP2, WP3 and WP4 outcomes in the EMERALD audit suite. • The methods to support the integration of pilots in WP5. This document is divided in three main parts. The first part presents an overview of the EMERALD framework. A context diagram has been included, showing the main inputs, outputs, and roles involved in the EMERALD workflow. Twelve different components of EMERALD are presented, as well as and the interaction among them. A Glossary of terms closes this part, where the definition of terms helps to understand the EMERALD context. The second part of the document presents the requirements elicited for the EMERALD framework. The requirements elicitation is an iterative process, mixing several perspectives, where Technical requirements (functional and non-functional), User Interface requirements and Pilot requirements are gathered independently. Afterwards, they are linked, integrated and analysed. We present the tools used to implement the process: GitLab Issues as the requirement definition and tracking tool; Component Cards template to describe components and PlantUML to the create the UML diagrams. Then, we describe the technical requirements elicited in the first 12 months of the project, grouped by components. They cover the expected functionalities of EMERALD framework. These are complemented by non-functional requirements, that cover a range of properties like performance, security, deployment, or availability, to cite some. These are system constrains which are transversal to many (or all) components. The pilot requirements, worked in WP5, are listed too, and then a mapping with the technical requirements has been presented. Next, an analysis of the requirements set has been performed, studying their relations, status, and coverage. For that, a set of traceability matrices shows the alignment of the elicited requirements with respect to the EMERALD Key Results, and which technical requirements implement a pilot requirement. To end, a prioritization matrix reflects which requirements will be implemented in each iteration of the EMERALD workplan. The last part of the document presents the EMERALD Framework detailed view, where each component is described in detail -functionality, interfaces, and behavioural modelusing the previously mentioned artifacts. The general data model is also included. Future version of this document is D1.4 [2], due at M24. It will provide and actualized set of requirements and their status, as design development tasks evolve. The next related task is the integration of the v1 version of the components into the first version of the integrated EMERALD framework, which will be produced in M18 of the project and reported in D1.7 [3]. D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 8 of 73 1 Introduction 1.1 About this deliverable This deliverable is the result of Task 1.2 – EMERALD architecture, in the WP1-Concept and methodology of EMERALD. Its main goal is to provide a common definition of the EMERALD Framework. The document includes an overview first, and a detailed description later of the EMERALD architecture. It describes the different components, modules, interactions and interfaces. A concise view of each component is presented, using a template named "Component Card", which contains key information about the component, such as: functionality, interfaces, subparts, and license. The component behaviour description is completed by UML sequence diagrams 1 , that show the interaction with the rest of components. The document provides a complete list of the technical requirements of the EMERALD CaaS framework. Part of them have been gathered and developed in cooperation with WP5 - that deals with the pilots’ implementation - and WP4 - which oversees the user experience and interaction in the EMERALD framework. Most of the requirements listed here have been already described in more detail in the deliverables of WP2 and WP3 (dedicated to describing the components in depth), WP4 (related to the UI) and WP5 (related to the pilots). An analysis of the requirements, their prioritization and status are also included. During the first year of the project, several workshops have been conducted among the work packages to coordinate the different views that stakeholders could have about what the EMERALD framework has to provide and how. One of the outcomes are the requirements gathered here. 1.2 Document structure The remainder of the document is organized as follows: Section 2 presents a global view of the EMERALD framework, its users and context. The section also includes a Glossary that captures the main terminology used in the project. Section 3 outlines the methodology and tools used in requirement management and documentation. The functional and non-functional requirements of the EMERALD Framework are presented, along with their priority and current status of implementation. A dashboard finalizes the section. Section 4 describes the architecture of the EMERALD CaaS framework. It provides a succinct description of the components that make up the EMERALD framework, their workflows, implemented interfaces, and sequence diagrams. Section 5 presents the conclusions, a summary of findings and outcomes. Finally, APPENDIX A: Current status of requirements contains the list of Technical requirements and their current fulfilment status. 1 https://en.wikipedia.org/wiki/Sequence_diagram D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 9 of 73 2 Overview of the EMERALD Framework This section contains the context diagram of EMERALD and the involved roles, introduces the framework, and provides a Glossary of the most relevant terms used in EMERALD. 2.1 Context diagram The context diagram of a system shows the roles involved, the basic workflow, as well as the inputs and outputs of the process. The roles that take part in the EMERALD ecosystem, as well as personas and scenarios, are being investigated in the workshops related to tasks T4.1 – Requirements engineering with compliance managers and auditors and T4.2 – Modelling work processes, in WP4. Table 1 summarizes the main roles in EMERALD. For more information on this subject, consult the deliverables D4.1 [4] and D4.3 [5]. Table 1. Roles in THE EMERALD ecosystem Generic Role Roles Description Compliance Stakeholders Compliance Manager Supports the company in being trustworthy, overseeing audit processes, being up to date with security standards, organizing audits and managing the scheduling of different compliance schemes. Creates an audit scope in EMERALD to manage the certification process. Compliance Manager for financial services Focuses on risk management of third-party cloud services, assesses controls based on risk and regulation, manages contractual agreements, and monitors compliance Metric Owner Their tasks consist of on defining metrics, collecting evidence for controls and assigning and delegating control implementation to Technical Implementers. NOTE: alternatively called Internal Control Owner Auditor Stakeholders Internal Auditor Reviews all controls of an audit scope. If some are noncompliant, checks the reasons and informs the Compliance Manager. External Lead Auditor In charge of managing the audit process, planning, reporting, and maintaining contact with customers. NOTE: both Auditors are a unique role in the EMERALD UI. External Technical Auditor Technical Stakeholder Technical Implementer Performs the technical tasks to implement an assigned control, through software development, configuration, etc. Selects a set of metrics that matches the controls, implements them, and informs the Metric Owner. NOTE: alternatively called Metric Implementor A first categorization divides the roles in three groups according to their function: (i) the Compliance Stakeholders (Compliance Managers and Metric Owner) that manage the certification process, organizing audits and preparing the system; (ii) the Auditor Stakeholders (Internal and External Auditors), that deal with the results of the assessment of an Audit Scope and report the result to the Compliance Manager; (iii) the Technical Stakeholder, who implements the required metrics for the Control owner. D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 16 of 73 3 EMERALD Framework Requirements In this chapter, we will list and jointly analyse the requirements gathered for the components during the first year of the project. During this time, several workshops have been maintained among the work packages to coordinate the different views that stakeholders could have about what the EMERALD framework must provide and how. One of the outcomes are the requirements presented here. Each requirement is uniquely identified by an ID, which will be referenced in future tasks and documents for prioritization, validation, etc. Please note that the requirements are not described in detail, i.e., using the template defined in Table 3, because they have already been described in detail in those deliverables describing the respective EMERALD component in WP2 (see D2.2 [13], D2.4 [14], D2.6 [15], D2.8 [16]), WP3 (see D3.1 [17]), WP4 (see D4.1 [4]) and WP5 (see D5.1 [18]). 3.1 Methodology and Tools for requirements elicitation In this section, we will briefly describe the methodology used in EMERALD for the elicitation of requirements and the principal tools and artifacts used to support the process. 3.1.1 The process The requirements gathering process followed in EMERALD is multi-focused. The process has been divided in three parallel paths, each one trying to investigate the EMERALD system from different perspectives. A first path that uncovers the functionalities and qualities that the technicians understand the EMERALD product has to offer. This work has been based in the documentation available: project proposal [19], key Results expected, norms and standards, and in the knowledge inherited from the MEDINA 3 project, which is the predecessor of EMERALD project. This path, carried under WP1, has produced a set of Technical requirements. These requirements have been covered in different deliverables in WP2 (D2.2 [13], D2.4 [14], D2.6 [15], D2.8 [16]) and WP3 (D3.1 [17]) devoted to describing the components. A second path has been devoted specifically to the user experience, to provide EMERALD with an advanced user interface that connects the rest of components and satisfies the users’ requirements while providing the needed information in its different views. This work has been conducted in WP4, where a co-design, participatory design approach has been followed, holding separate interviews with component owners and with pilot owners. This has produced a set of User Interface requirements (more information on this is available in D4.1 [4]). Lastly, a third path has been focused on what the final users of EMERALD have asked to be part of the delivered product. This work has been part of WP5, where the pilots have been defined, and has produced a set of Business requirements (more information on this is available in the deliverable D5.1 [18]). All these separate elicitations have produced separate requirement sets. One of the tasks in WP1 has been to analyse, refine and check these requirements, approve the correct ones and discard others, as well as to establish the relationships among them. Several discussions about the requirements have hold during the periodic work package meetings. Also, specific workshops have been conducted to map the business requirements and user interface 3 https://medina-project.eu/ D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 17 of 73 requirements to technical requirements. This has produced changes in the requirements, and new requirements have been defined when necessary. This document presents the results of this analysis, managing the different lists of requirements. We provide a dashboard with the status and prioritization of requirements. Furthermore, several traceability matrixes are presented, to keep all the relationships affecting the requirements up to date. 3.1.2 The tools To carry out the architecture definition, different tools and artifacts have been used, namely Gitlab issues, Component cards and UML models with PlantUML tool, that will be briefly described in the following. 3.1.2.1 Gitlab issues To better control their changes and evolution, the requirements in EMERALD have been defined in GitLab, using the issues 4 feature. Issues are used in general to collaborate on ideas, solve problems, and plan work. They allow to track tasks and work status, accept feature proposals, ask questions, or support requests. A template has been used to define the requirements, as depicted in Table 3. The template has a tabular form and contains all the fields needed to gather the requirement information and track it during the project lifetime. The table has been also implemented as a GitLab template, useful to define new requirements. Table 3. Requirement template Field Description Requirement ID Unique identifier. E.g., for the Repository of Controls and Metrics -> RCM.01, RCM.02… Short title Short description of the requirement Description More detailed description of the requirement. This is especially relevant for the creation of the test cases. Status Choose the corresponding label: Status::Proposed -> Status::Accepted / Status::Discarded -> Status::Work in Progress -> Status::Implemented -> Status::Validated Priority Choose the corresponding label: Priority::Must -> Priority::Should -> Priority::Could Component Choose the corresponding label: Comp::AI-SEC, Comp::AMOE, Comp::CertGraph, Comp::Clouditor, Comp::Codyze, Comp::eKnows, Comp::EmeraldUI, Comp::EvidenceStore, Comp::LCM, Comp::RCM, Comp::RMA, Comp::TWS, Comp::WP1, Comp::N/A Source Pilots / Component / DoA / KPI Type Choose the corresponding label: Choose the corresponding label: Type::Technical, Type::Pilots, Type::GUI 4 https://docs.gitlab.com/ee/user/project/issues/ D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 18 of 73 Related KR Choose the corresponding label: KR::KR1_EXTRACT, …, KR::N/A Related KPI Choose the corresponding label: KPI::1.1, …, KPI::N/A Validation acceptance criteria Describe how to validate the requirement. What are the steps to follow, what should be the system output Progress [Optional] percentual degree of advances from 0% to 100% Milestone Select the milestone among the defined ones: from MS1: Components V1 (M12) to MS9: Final evaluation report and impact analysis (M36) As mentioned above, this table has been used in other WP2 and WP3 deliverables dedicated to describing the components in detail. In this document, we will mainly limit to listing the requirements and analysing them as a whole. Figure 3 shows a list of the requirements in the GitLab requirements repository. The developer can define a new requirement using the aforementioned template. To facilitate requirements identification and filtering, a set of labels associated to the issues have been defined. Labels are organized in categories, where each category defines a property of the requirement and is represented in different colours. Categories for labels are: • Component label (one for each component) • Type label (Technical / Pilots, UX) • Priority label (Must / Should / Could) • KR label (one for each Key Result) • Pilot label (Ionos / CloudFerro / Fabasoft / Caixabank) • Status label (Proposed / Accepted / Discarded / Implemented / Validated) • KPI label (one for each Key Performance Indicator) Requirements can be filtered using lists or also be visualized and managed using issue boards 5 of GitLab. The issue board is a software management tool used to plan, organize, and visualize a workflow for a feature or product release, pairing issue tracking and project management. The boards organize the issues in cards, in vertical lists organized by their labels, milestones, or assignees. Requirements can be managed inside the boards. For example, moving a requirement from one list to other changes the associated label and thus the requirement properties. Several specific boards have been defined in EMERALD to provide different views of the requirement set: • Requirements by TYPE(Technical/GUI/Pilots) • Requirements by PRIORITY(Must/Should/Could) • Requirements by KR • Requirements by STATUS • Requirements by COMPONENT • Requirements by Pilot 5 https://docs.gitlab.com/ee/user/project/issue_board.html D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 19 of 73 Figure 3. List of requirements as issues in GitLab (excerpt) 3.1.2.2 Component cards A “component card” is what we call a piece of information that contains a brief description of each component. It contains the essential information to know what the component does, where it fits in the framework, with which other components it interacts and how it is made. A component card has been defined for each component, and all of them are included as part of the detailed view of the EMERALD framework in Section 4. Table 4 shows the structure of a component card. Table 4. Component card template Component Name Name of the component and acronym, if any Main functionalities List the main functionalities the component provides. E.g.: • Describe functionality 1 • Describe functionality 2 Subcomponents Description Subcomponent A: Describe the functionality of the sub-component Subcomponent B: Main logical Interfaces offered Include graphical interfaces if any. Interface name Description Interface technology D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 20 of 73 Interaction with other components • Component X: Describe interaction with component X • Component Y: Relevant sequence diagram/s Include a shot of the sequence diagram(s) describing the component’s dynamic behaviour Requirements Mapping List the requirements covered by this component. E.g.: • TWS.01: Provide integrity proof of evidence • TWS.02: Technology used Describe the technology used in the implementation of the component (languages, frameworks, etc) Related KR Related EMERALD proposal Key Results WP and task WPX – Tx.1 License License of the component Partner Partner that is the component owner, who defines/implements it. 3.1.2.3 PlantUML diagrams Diagrams of the Unified Modelling Language (UML) have been used in the definition of the EMERALD architecture. More concretely, Class diagrams to define the data model the components use, and the relationship among the objects; and Sequence diagrams, to define the dynamic behaviour of the components and the flow of information among them. This kind of diagram visualizes the interactions between users, systems and sub-systems over time, through message passing between objects or roles. UML sequence diagram complete the classes or object diagram, that represent the attributes, by representing the programming logic to be filled in the methods’ body. To define the UML diagrams, the PlantUML 6 tool was chosen. This tool creates the diagram based in text descriptions and supports a wide range of diagrams. PlantUML allows to render the diagrams as images in different output formats. As the PlantUML based diagrams contain text/code, the files are included in Gitlab for versioning. This allows for different organisational processes, that are not possible in common online tools with graphical support. New versions of the diagrams are produced with each commit, and merge requests are created to change the actual release. As the specific diagram for each component has been included in the deliverable D1.1 [1], in this document we only present a general class diagram representing the whole EMERALD framework. However, sequence diagrams for each component are included in Section 4 as part of the detailed view of the EMERALD framework. 3.2 Functional Requirements Table 5 lists the set of functional requirements of the EMERALD framework components. Along with the brief description, the priority and milestone of each requirement are presented. A total of 44 functional requirements have been elicited, grouped in the 12 components that form the framework. 6 https://plantuml.com/ D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 21 of 73 The Identification of each requirement is unique. It is composed by the acronym of the component plus a number. The components have been described in Section 2.2, but a list with its correspondence to Identifiers is provided below for clarity. • AI-SEC: AI Security Evidence Collector • AMOE: Assessment and Management of Organisational Evidence • CLDISC: Clouditor-Discovery • CODYZE: Codyze • EKNOWS: eknows - Software analysis platform • TWS: Trustworthiness System • MARI: Mapping Assistant for Regulations with Intelligence • RCM: Repository of Controls and Metrics • ORCH: Clouditor-Orchestrator • ESTORE: Clouditor-Evidence Store • ASSESS: Clouditor-Assessment • EVAL: Clouditor-Evaluation The Milestone field of each requirements signals when the requirement is foreseen to be completed. The list of Milestones corresponds to the ones defined in the DoA: • MS1: Project baselines and definition (M9) • MS2: Components V1 (M12) • MS3: Integrated audit suite V1 (M18) • MS4: Pilots V1 (M20) • MS5: Components V2 (M24) • MS6: Integrated audit suite V2 (M30) • MS7: Pilots V2 (M32) • MS8: Integrated audit suite V3 (M34) • MS9: Final evaluation report and impact analysis (M36) Table 5. Functional requirements. Req. ID Description Priority Milestone AI-SEC.01 The extractor tool includes defined criteria: The designed AISEC has the selected criteria of the BSI AIC4 Must MS2 (M12) AMOE.01 Upload PDF document: The component shall be able to receive a PDF document via API and process its contents regarding the defined metrics. The PDF shall receive a unique ID so that it can be retrieved and deleted later on. Must MS2 (M12) AMOE.02 Provision of extracted evidence to EvidenceStore: The evidence extraction component needs to be able to forward the extracted evidence to the EMERALD EvidenceStore, so it can be used for assessment and further audit processes. Must MS5 (M24) AMOE.03 Refine evidence extraction approach: The evidence extraction approach should be refined to the needs of the pilots, so that the tool is able to provide relevant evidence for the metric assessments. Must MS5 (M24) AMOE.04 Compare results from multiple documents: Results from multiple policy documents shall be comparable using AMOE. A metric can be used to extract evidence from different policy documents. AMOE shall provide the results via API for a metric and given cloud service. Should MS2 (M12) D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 22 of 73 Req. ID Description Priority Milestone AMOE.05 Select metrics per document: AMOE should offer the possibility to select some metrics before they are extracted for a document. This speeds up the processing time as metrics that are not contained in the document do not need to be checked. Also, it should be more convenient for the user, as the results are more precise and less irrelevant results need to be discarded. Should MS5 (M24) AMOE.06 Classify document, select respective metrics (optional): AMOE could use document classification to pre-select some metrics based on the category, text, requirements or other feature that would be of use. This could potentially, reduce the manual workload and help to provide only results for metrics that target the specific document. Must MS8 (M34) AMOE.07 Metric states: AMOE could add some internal states to the metrics. This should help to visualize the current process for every metric and role. Here is a list of metric flags that could be used: new, internal-started, ready-for-audit, revise-policy, audit-finished, result-outdated, extraction-failed. - new: the metric has been successfully extracted - extraction-failed: evidence could not be extracted - internal-started: internal auditor/compliance manager started inspecting the metric - ready-for-audit: internal auditor/compliance manager has finished with the metric, and marked it ready for auditor - revise-policy: auditor sets metric to be revised - audit-finished: auditor is ok with metric - result-outdated: automatic or manual triggered check if result is outdated Should MS5 (M24) CLDISC.01 Discovery of security properties of infrastructure components: The Clouditor discovery needs to discover security properties of infrastructure components. The evidence with the security properties is sent to the Evidence Store in the ontology format. Must MS6 (M30) CODYZE.01 Extraction of security features from source code: Codyze needs to check available source code artefacts for security features. Must MS6 (M30) EKNOWS.01 Integration into existing systems: The component should be integrable into existing systems, development environments and workflows, for example by using APIs like REST by compatibility with CI/CD-Pipelines. Must MS3 (M18) EKNOWS.02 Resilience while analysing erroneous code: The source code analysed by the component could be erroneous, for example syntactical and semantical errors could be encountered while parsing it. Furthermore, an unknown dialect of a language could be encountered. An appropriate error handling strategy for such situations is necessary: Erroneous code will be skipped and not be further analysed. A corresponding error message will be stored in the gathered evidence. Should MS5 (M24) EKNOWS.03 Multi-language support: The component should be able to analyse source code written in different programming languages and should support at least Java and Python. Must MS5 (M24) EKNOWS.04 Support EMERALD evidence format: The analyzation results are offered in a structured and standardized format, the EMERALD evidence format (see data model). This enables further processing and queries in other components. Must MS3 (M18) D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 23 of 73 Req. ID Description Priority Milestone EKNOWS.05 Static code analysis: The component uses static code analysis methods. Such methods are, for example, data flow analysis, call graph analysis, symbolic execution or control flow analysis. One or multiple methods (possibly in combination) will be used to gather evidence. The actual used method(s) depend(s) on the metric, for which evidence should be extracted. Must MS5 (M24) TWS.01 Provide a tool allowing the verification of evidence integrity without needing to store the evidence itself (for confidentiality reasons). Must MS2 (M12) TWS.02 Provide a tool allowing the verification of assessment results integrity without needing to store the result itself (for confidentiality reasons). Must MS2 (M12) TWS.03 The integrity validation of evidence and assessment results must be done through REST API or graphical interface (EMERALD UI). Must MS5 (M24) TWS.04 The TWS must be based on a real Blockchain network, with multiple nodes and multiple organizations to guarantee suitable decentralization and governance of the Blockchain network. Must MS5 (M24) MARI.01 AI-based: MARI is a tool based on state-of-the-art artificial intelligence, e.g., uses a transformer-based architecture Must MS6 (M30) MARI.02 Automatic association: MARI takes as input cloud security controls written in natural language, metrics that validate those controls, again written in natural language, and automatically returns as output the association control/metric(s) and the association control/control. Must MS6 (M30) MARI.03 Performance Evaluation: The performance of MARI should improve on the performance of the Metric Recommender of EMERALD’s predecessor project, MEDINA. We can assume that we measure the performance of MARI with the same metrics used for the Metric Recommender, namely precision@k and NDCG (Normalised Discounted Cumulative Gain) Must MS6 (M30) MARI.04 Usage and Visualization: MARI should be invoked through EMERALD's built-in interface, and MARI results can be visualized through the same interface Must MS6 (M30) MARI.05 Strategies: MARI can act according to specific strategies, such as considering only technical controls, or organizational controls, or controls of a certain category, or controls whose implementation costs less in terms of human resources, etc. The strategies will be defined during the project. Must MS6 (M30) RCM.01 Multi-schema support: The repository should contain at least an additional security scheme, apart from the EUCS that is the scheme implemented in MEDINA Catalogue and is inherited in EMERALD Must MS2 (M12) RCM.02 Accessible by the rest of components: The repository content should be made accessible to the rest of EMERALD components via API Must MS2 (M12) RCM.03 Include metrics for all schemes supported: The repository should include metrics that could be used to assess the compliance with one or more certification schemes Must MS2 (M12) RCM.04 Mapping of schemes: The repository should support the mapping of the certification schemes contained. The schemeto-scheme mapping will be provided by the MARI tool and stored in the repository. The rationale for the mapping decision will also be stored Should MS5 (M24) D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 24 of 73 Req. ID Description Priority Milestone RCM.05 Import/export of security schemes in OSCAL: The repository is able to import a new scheme defined in the OSCAL language (this feature can also be used to update an existing scheme). The repository is able to export any available scheme in OSCAL format Must MS6 (M30) RCM.06 Import/export of security schemes in CSV format: The repository can export a scheme to a CSV file, and import a CSV file with the same format as a new scheme Could MS2 (M12) RCM.07 Support for personalized catalogues: The Repository has to offer the user the possibility to create a personalized catalogue of controls. These controls can be taken from the same or from different security schemes Must MS6 (M30) RCM.08 Support updating/versioning of schemes: The Repository has to maintain a versioning system of the schemes it contains, so that if a new version is uploaded, it is able to detect the change and notify the user that a new version is available Should MS6 (M30) ORCH.01 Final certificate decision: Since we do not have a dedicated life-cycle manager component in EMERALD, the Orchestrator must take care of the final certificate decision. The decision is based on the input of the Evaluation component providing the Orchestrator with an evaluation result for each control Must MS5 (M24) ORCH.02 REST API Gateway for UI: The Orchestrator should provide a REST API gateway for the UI that serves a central API endpoint for all information needed from the Orchestrator, Assessment, Evaluation and other Clouditor components. Must MS2 (M12) ORCH.03 Role Based Access Control (RBAC): Since the UI wants to selectively disclose information to users and/or roles, we need a RBAC mechanism in our API endpoints, mainly in the Orchestrator. Must MS5 (M24) ORCH.04 Manage Tools via API: We need to manage external tools, such as evidence extractors in the Orchestrator. Should MS5 (M24) ORCH.05 Provide an API for audit workflow: We want to assign people to controls within an audit instance that have a particular task. Must MS6 (M30) ESTORE.01 Storage of evidence as ontology entities in graph database: The Evidence Store must store the evidence according to the schema defined by the knowledge graph. The preferred way to store this information is a graph database. Must MS3 (M18) ESTORE.02 Allow Interaction with Third-Party Tools: The Evidence store should be allowed to accept evidence from third-party tools, e.g., using a REST API. The evidence needs to be in the ontology format. Therefore, information about the ontology and data models must be available. Should MS3 (M34) ASSESS.01 Assessment based on evidence: The assessment should assess evidence based on the knowledge graph. Must MS6 (M30) ASSESS.02 Assessment rules for 80% of the defined metrics: Assessment rules must exist for 80% of the metrics defined in KPI4.1. Must MS6 (M30) ASSESS.03 Display cause of assessment result: We want to know why an assessment result fails or passes. Could MS6 (M30) EVAL.01 Display cause of failing evaluation result: We want to know why the evaluation result fails or passes. Therefore, it should contain a list of assessment results that cause the evaluation status to be non-compliant. Could MS6 (M30) EVAL.02 Evaluation based on assessment results: The evaluation should assess the result based on all the required assessment results stored in the database. Must MS6 (M30) D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 25 of 73 3.3 Non-Functional Requirements The technical requirements presented in Section 3.2 involve behavioural, or functional, requirements of the system. They tell us how the system must behave when presented with certain inputs or conditions. But, in addition to these functional requirements, we have defined some non-functional requirements for the EMERALD framework. The following subsections provide different types of non-functional requirements, gathered in different work packages. 3.3.1 Other WP1 requirements We present here a list of nonfunctional requirements defined in WP1. These requirements are related with characteristics or constrains of the system more that to its behaviour. They have not been included in any previous deliverables, so we follow each requirement with a short paragraph on how we plan to implement it. Requirement id WP1.01 Short title Performant framework Description The EMERALD framework should be as performant as possible. The response time for a user action in normal conditions should not be larger than a few seconds. Implementation state Partially implemented The component tools will have to pass automatic integration tests by the CI/CD pipeline before being integrated into the framework. The validation task in WP5 will validate both the functionality and the performance of the EMERALD framework. Apart of these controls, the framework infrastructure is continuously monitored, and the implemented environment allows flexibility to upgrade the resources if they are falling short (e.g., adding more memory or CPUs to the Kubernetes nodes, or providing extra nodes). Requirement id WP1.02 Short title Portability Description The EMERALD framework should be portable and work in any typical business environment. Implementation state Partially implemented The components of the framework will be packaged as containers, which are a portable technology by definition. We will use the Docker ecosystem to build and share images. For image building we will support both Docker and Docker Compose. Requirement id WP1.03 Short title Scalability Description The EMERALD framework should be easily scalable when the working conditions become severe in relation to the number of users of the platform or intense use. Status Partially implemented Scalability will be based in the use of a container orchestration technology, such as Kubernetes, which is inherently scalable. It also can provide resilience, helping to solve problems when the resources allocation is shorter that needed. Requirement id WP1.04 Short title Installability D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 32 of 73 • KPI 2.3: Provide scalability for storing/processing continuously collected evidence; demonstrated in the pilots • KPI 3.1: Provide scheme to scheme mapping functionality based on metrics, recommended to the user • KPI 3.2: Provide metric-to-requirement-mapping functionality by improving MEDINA approaches and incorporating KPI 5.1 results • KPI 3.3: Provide insights for the mapping decision and how the recommendation process works • KPI4.1: Provide realizable metrics that demonstrate compliance to at least two security certification schemes • KPI 4.2: Provide metric assessment for 80 % of the metrics in KPI 4.1 based on the certification graph • KPI 5.1: Provide realizable metrics to help evaluate at least 50% of the categories of criteria of the BSI AIC4 that deal with the robustness of ML system, their interpretability, and the mitigation of potentially negative impacts such as model unfairness (c.f. Chapter 6, AIC4). • KPI 5.2: Provide a PoC for semi-automated assessment of 80% of the metrics specified in KPI 5.1. • KPI 6.1: Provide roles and workflows, derived from interviews with relevant users (e.g., project partners and advisory board members), develop mock-ups and interaction concepts for managing the audit process • KPI 6.2: Provide concept for the (UI) of EMERALD and integration of evidence collection components, data bases and orchestrating components • KPI 6.3: Provide a graphical user interface for role-based access to certification information content • KPI 7.1: Conventionalize import and export functionalities to take or share data with external sources • KPI 7.2: Incorporate input from standardisation bodies and synchronize data formats and protocols • KPI 8.1: Facilitate at least two different audit scenarios, one for public clouds, one for private cloud installations • KPI 8.2: Validate user acceptance in terms of complexity reduction Table 9. Functional requirements and KPIs alignment matrix. Req. ID EXTRACT CERTGRAPH OPTIMA M-CERT AIPOC UI/UX INTEROP PILOTS KPI1.1 KPI1.2 KPI2.1 KPI2.2 KPI2.3 KPI3.1 KPI3.2 KPI3.3 KPI4.1 KPI4.2 KPI5.1 KPI5.2 KPI6.1 KPI6.2 KPI6.3 KPI7.1 KPI7.2 KPI8.1 KPI8.2 AI-SEC.01 X X X AMOE.01 X AMOE.02 X AMOE.03 X AMOE.04 X AMOE.05 X AMOE.06 X AMOE.07 X D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 33 of 73 Req. ID EXTRACT CERTGRAPH OPTIMA M-CERT AIPOC UI/UX INTEROP PILOTS KPI1.1 KPI1.2 KPI2.1 KPI2.2 KPI2.3 KPI3.1 KPI3.2 KPI3.3 KPI4.1 KPI4.2 KPI5.1 KPI5.2 KPI6.1 KPI6.2 KPI6.3 KPI7.1 KPI7.2 KPI8.1 KPI8.2 CLDISC.01 X CODYZE.01 X EKNOWS.01 X EKNOWS.02 X EKNOWS.03 X EKNOWS.04 X EKNOWS.05 X TWS.01 X TWS.02 TWS.03 TWS.04 X MARI 1.0 X X X MARI 2.0 X X X MARI 3.0 X X X MARI 4.0 X X X MARI 5.0 X X X RCM.01 X X RCM.02 RCM.03 X X RCM04 X X RCM.05 X X RCM.06 X RCM.07 X X RCM.08 X X ORCH.01 X X ORCH.02 X ORCH.03 X ORCH.04 X ORCH.05 ESTORE.01 X X ESTORE.02 X ASSESS.01 X X ASSESS.02 X ASSESS.03 D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 34 of 73 Req. ID EXTRACT CERTGRAPH OPTIMA M-CERT AIPOC UI/UX INTEROP PILOTS KPI1.1 KPI1.2 KPI2.1 KPI2.2 KPI2.3 KPI3.1 KPI3.2 KPI3.3 KPI4.1 KPI4.2 KPI5.1 KPI5.2 KPI6.1 KPI6.2 KPI6.3 KPI7.1 KPI7.2 KPI8.1 KPI8.2 EVAL.01 EVAL.02 It can be seen from Table 9 that some of the KPIs are not directly addressed by any technical requirements. But this does not mean they are not covered by the EMERALD framework. In fact, they are generic KPIs that affect the whole framework and are addressed in a holistic manner. These are the KPIs in question (coloured in the table): • KPI 6.1 (related to providing roles and workflows, develop mock-ups for the audit process): It is closely related with all the work being carried in the WP4, where an UI/UX design process with stakeholders is leading to the definition of the roles and a set of mock-ups. • KPI 8.1, KPI 8.2 (related with pilots’ implementation and validation): This aspect is being covered by the WP5, where the pilots have been designed and, in general, the whole EMERALD framework is covering them. 3.4.3 Mapping of requirements to Business Driven Requirements In the end, the business-driven requirements (BDRs) must be implemented in the components. To ensure the technical implementation, the business-driven requirements were reviewed in collaboration with WP5 in joint workshops and mapped to technical requirements. This work assigns a list of component technical requirements to each business-driven requirement. The alignment in Table 10is intended to show that each BDR defined by the Pilots has one or more corresponding components that implement it. In this way, a Pilot can identify the component responsible for implementing each BDR and track its coverage along the time. A BDR with no associated functional requirements means that it is either out of scope of the EMERALD framework -as it is currently definedor that the framework doesn’t contemplate all the user needs. In the latter case, this table will serve for components designers to identify missing functionalities from the Pilots perspective, thus aligning both perspectives used for the elicitation of the functional requirements. Table 10. Technical requirements vs Business Requirements alignment matrix. Req. ID Pilot 1 Ionos Pilot 2 Cloudferro Pilot 3 Fabasoft Pilot 4 Caixabank BDRP1.01 BDRP1.02 BDRP1.03 BDRP1.04 BDRP1.05 BDRP1.06 BDRP1.07 BDRP2.01 BDRP2.02 BDRP2.03 BDRP2.04 BDRP2.05 BDRP3.01 BDRP3.02 BDRP3.03 BDRP3.04 BDRP3.05 BDRP3.06 BDRP3.07 BDRP3.08 BDRP3.09 BDRP3.10 BDRP3.11 BDRP3.12 BDRP4.01 BDRP4.02 BDRP4.03 BDRP4.04 BDRP4.05 BDRP4.06 BDRP4.07 AI-SEC.01 X X AMOE.01 X X X AMOE.02 X X X AMOE.03 X D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 35 of 73 Req. ID Pilot 1 Ionos Pilot 2 Cloudferro Pilot 3 Fabasoft Pilot 4 Caixabank BDRP1.01 BDRP1.02 BDRP1.03 BDRP1.04 BDRP1.05 BDRP1.06 BDRP1.07 BDRP2.01 BDRP2.02 BDRP2.03 BDRP2.04 BDRP2.05 BDRP3.01 BDRP3.02 BDRP3.03 BDRP3.04 BDRP3.05 BDRP3.06 BDRP3.07 BDRP3.08 BDRP3.09 BDRP3.10 BDRP3.11 BDRP3.12 BDRP4.01 BDRP4.02 BDRP4.03 BDRP4.04 BDRP4.05 BDRP4.06 BDRP4.07 AMOE.04 X X AMOE.05 X AMOE.06 X AMOE.07 X CLDISC.01 X CODYZE.01 X X X EKNOWS.01 X EKNOWS.02 EKNOWS.03 EKNOWS.04 EKNOWS.05 X TWS.01 X X X X X TWS.02 X X X X X TWS.03 X X X TWS.04 X X X X MARI 1.0 X X X X X X X MARI 2.0 X X X X X X X X X MARI 3.0 MARI 4.0 X X MARI 5.0 X X X X X X X RCM.01 X X RCM.02 X X RCM.03 X RCM04 X X X RCM.05 X X X X RCM.06 X X RCM.07 X X X RCM.08 X X ORCH.01 X X X X X X X ORCH.02 X X X X X X X ORCH.03 X X ORCH.04 X X X X D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 36 of 73 Req. ID Pilot 1 Ionos Pilot 2 Cloudferro Pilot 3 Fabasoft Pilot 4 Caixabank BDRP1.01 BDRP1.02 BDRP1.03 BDRP1.04 BDRP1.05 BDRP1.06 BDRP1.07 BDRP2.01 BDRP2.02 BDRP2.03 BDRP2.04 BDRP2.05 BDRP3.01 BDRP3.02 BDRP3.03 BDRP3.04 BDRP3.05 BDRP3.06 BDRP3.07 BDRP3.08 BDRP3.09 BDRP3.10 BDRP3.11 BDRP3.12 BDRP4.01 BDRP4.02 BDRP4.03 BDRP4.04 BDRP4.05 BDRP4.06 BDRP4.07 ORCH.05 X X X X X X ESTORE.01 X X X ESTORE.02 X X ASSESS.01 X ASSESS.02 X ASSESS.03 X X X EVAL.01 X X EVAL.02 X As mentioned above, each BDR to be implemented should be related to at least one component. Otherwise, it would mean that no component is implementing such requirement. According to the Table 10, the BDRs that fall into this category are the following: • BDRP1.07 - Intuitive User Experience for Compliance Monitoring: this requirement is addressed by all the UI/UX requirements developed in WP4. • BDRP3.07 - Enhance current audit process: It is a very generic requirement that must be addressed by the whole EMERALD platform, as all components are involved in the improving the audit process. 3.4.4 Prioritization and current status Table 11 depicts the status of the functional requirements foreseen for M12 (at milestone MS2: Components V1), the due date of this deliverable. For a complete table with the status of all requirements, view the APPENDIX A: Current status of requirements. Table 11. Requirements prioritization matrix Req. ID Title Priority Timeline Status AI-SEC.01 The extractor tool includes selected criteria MUST M12 (C-v1) 35% AMOE.01 Upload PDF document MUST M12 (C-v1) 90% AMOE.04 Compare results from multiple documents SHOULD M12 (C-v1) 70% TWS.01 Provide integrity proof of evidence MUST M12 (C-v1) 75% TWS.02 Provide integrity proof of assessment results MUST M12 (C-v1) 75% RCM.01 Multi-schema support MUST M12 (C-v1) 90% RCM.02 Accessible by the rest of components MUST M12 (C-v1) 100% RCM.03 Include metrics for all schemas supported MUST M12 (C-v1) 30% RCM.06 Import/export of security schemes in CSV format COULD M12 (C-v1) 60% D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 37 of 73 Req. ID Title Priority Timeline Status ORCH.02 REST API Gateway for UI MUST M12 (C-v1) 15% 3.5 Requirements Summary Dashboard Table 12 shows a summary of requirements by component, with their status -in a broad vision divided in not started, partially implemented and fully implementedat the moment of writing. Table 12. Summary table of requirements status at M12 (by component) Component Not started Partially implemented Fully implemented TOTAL AI-SEC 0 1 0 1 AMOE 4 3 0 7 Discovery 0 1 0 1 Codyze 0 1 0 1 eKnows 1 4 0 5 TWS 0 4 0 4 MARI 0 5 0 5 RCM 1 6 1 8 Evidence Store 0 2 0 2 Orchestrator 3 2 0 5 Assessment 1 2 0 3 Evaluation 1 1 0 2 NFR (WP1) 0 7 1 8 TOTAL 11 39 2 52 It can be observed that, because of the different ranges of functionality of each component, the requirements are not equally distributed among the components (see Figure 4). It is also the case that not all components have yet the same level of definition. In this respect, the components with the most requirements are RCM (with 8), AMOE (with 7) and MARI, Orchestrator and eknows (with 5 each). D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 38 of 73 Figure 4. Number of requirements per component Regarding the status of the requirements at M12 (see Figure 5), most of them are in a work in progress status (32 out of 52); the not-started requirements are half of the started ones (16 out of 52); and few requirements are already fully implemented (4 of 52). Figure 5. Requirement status Figure 6 shows the status of requirements by component. Logically, the same pattern that in the overall view can be observed, i.e., all the components have a majority of partially implemented requirements, with some requirements nor yet started and only a few completed requirements. Not started; 21% Partially ; 75% Fully impl.; 4% Requirement status Not started Partially Fully impl. D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 39 of 73 Figure 6. Requirement status per component Finally, let’s have a look to the coverage of the requirement sets to the different pilots. Table 13 shows the number of requirements for each component that cover some aspect of each pilot (a pilot requirement). We can see that the most covered pilot is Pilot 4, with 50 requirements, followed by Pilot 3 (38), Pilot 2 (17) and Pilot 1 (16). The colour shows, for each pilot, which component contributes the most (red), with the intensity decreasing as the contribution of the component to the pilot decreases. Table 13. GENERAL VIEW: Components vs Pilot Component Pilot 1 Pilot 2 Pilot 3 Pilot 4 TOTAL AMOE 3 0 5 4 12 MARI 3 5 12 5 25 RCM 4 2 7 4 17 TWS 3 4 3 7 17 Cloud. Assessment 0 1 2 2 5 Cloud. Discovery 0 1 0 0 1 Cloud. Evaluation 2 0 0 1 3 Cloud. Evidence Store 0 0 2 3 5 Cloud. Orchestrator 3 2 3 18 26 Codyze 1 0 0 1 2 eKnows 1 0 0 1 2 AI-SEC 0 0 1 1 2 NFR 1 0 0 6 7 D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 40 of 73 4 EMERALD Framework detailed view This section describes the architecture of the EMERALD CaaS framework. It provides a succinct description of the components that make up the EMERALD framework, their workflows, implemented interfaces, and sequence diagrams. 4.1 Data model The EMERALD data model was defined in D1.1 [1], that describes the different data classes used by the components, and the connections within and between components. The data model is useful mainly for the developers of the EMERALD framework in order to construct the software classes to manage the required data structures. The data model for the whole EMERALD framework is shown in Figure 7, where each component is represented in a box, that includes inside the data structures it handles. The background colour of the box denotes the project work package to which the component pertains. Thus, Evidence Collection components (WP2) are coloured in orange, whereas WP3 components are coloured in teal. The EMERALD project uses some of the components that were part of the MEDINA data model – such as the Evidence Store, the Orchestrator, the Repository of Controls and Metrics (RCM) and the Trustworthiness System. D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 41 of 73 www.emerald-he.eu Figure 7. EMERALD data model (D1.1 [1]) D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 48 of 73 www.emerald-he.eu Figure 11. Clouditor-Discovery sequence diagram 4.2.1.4 Codyze Component Name Codyze Main functionalities The component provides the following functionalities: • Scans source code for insecure implementations of securityrelevant features (e.g., transport encryption, logging, authentication & authorisation, etc.) • Analyse interactions between cloud service components from infrastructure-as-code (e.g., What cloud resources are consumed?, Are interactions secure?, Are used resources up-to-date?, etc.) • Analyse development processes (e.g., Are secure development processes followed?, Is the provenance of source code guaranteed?, What measures are taken to secure the development pipeline?, etc.) Subcomponents Description Currently no division in subcomponents planned Main logical Interfaces offered Interface name Description Interface technology CLI A CLI incl. configuration file to configure Codyze and set execution/analysis parameters. Kotlin Clikt library16 Interaction with other components • Orchestrator o Request information on cloud service to be analysed • Evidence Store o Submit evidence to be stored 16 https://ajalt.github.io/clikt/ D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 49 of 73 www.emerald-he.eu Relevant sequence diagram/s See section 4.2.1.4.1 Requirements Mapping List of requirements covered by this component: • CODYZE.01: Extraction of security features from source code Technology used Kotlin17 Related KR KR1 WP and task WP2 – T2.2 License Apache-2.0 Partner Fraunhofer AISEC 4.2.1.4.1 Sequence diagram Figure 12 shows the sequence diagram of the Codyze component. Codyze provides evidence extraction from source code of cloud services. It analyses and generates evidence results that indicate if code segments are compliant or non-compliant to specified requirements. These evidence results are submitted to the Evidence Store for storage and further processing. As in the case of AI-SEC, it is recommended to run it as part of a CI/CD pipeline, that prevents the deployment of non-compliant services and application. For that, some initial configuration is needed. Figure 12. Codyze sequence diagram 17 https://kotlinlang.org/ D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 50 of 73 www.emerald-he.eu 4.2.1.5 eknows Component Name eknows Main functionalities The component provides the following functionalities: • Static code analysis. • Language-independent frontends (currently >16 programming languages, including Java, Python, Cobol, C++, etc.). • Rapid development platform for software tools such as documentation generators and tools for reverse engineering and code visualization. • Extraction of business rules from code. Subcomponents Description eknows is a Java-based software platform to build reverse engineering tools and documentation generators. The platform provides a modular extensible set of software components, which facilitate the rapid development of tools in program comprehension, documentation generation, and software reverse engineering. Support for multiple programming languages in terms of language-specific extraction components and language-independent analysis is a key feature of the platform. The platform (see Figure 13) provides reusable components that facilitate (i) language parsing (extraction), (ii) transformation of source code into a generic abstract syntax tree (GASTM), (iii) structural and behavioural analysis of software, and (iv) reporting and visualization of analysis results. Figure 13. Overview of eknows platform components Tools built on top of eknows integrate required software components as-is and add functionality required for a specific use case. Main logical Interfaces offered Interface name Description Interface technology Java API eknows can be added as a set of Java libraries (eknows-core, eknows-frontends, eknowsanalysis, etc.) to call its components. Java REST (maybe) The analyzation of source code files can be triggered via a REST endpoint. HTTP / REST D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 51 of 73 www.emerald-he.eu CLI The analyzation of source code files can be triggered via a command line interface. stdin/stdout Note: REST interface does not exist yet, however, if needed, it will be developed within EMERALD. Interaction with other components • Evidence Store: Sends (raw) evidence. • CI/CD Pipeline: Starts analyzation of source code files by calling a trigger provided by eknows. Relevant sequence diagram/s See section 4.2.1.5.1 Requirements Mapping The requirements covered by this component are: • EKNOWS.01 – Integration into existing systems • EKNOWS.02 – Resilience while analysing erroneous code • EKNOWS.03 – Multi-language support • EKNOWS.04 – Support EMERALD evidence format • EKNOWS.05 – Static code analysis Technology used Java Ecosystem Related KR KR1 EXTRACT WP and task WP2 – T2.2 License eknows-core, reused frontends and reused analyses eknows Binary Usage Software License eknows extractor Apache License, Version 2.0 Partner SCCH 4.2.1.5.1 Sequence diagram Figure 14 shows the sequence diagram of the eknows component. eknows supports the creation of evidence extraction functions by reusing prefabricated parsing, analysis, and generation modules, with the mission to verify if application source code complies to security requirements. eknows can be integrated into CI/CD pipelines by using the binary distribution. Findings are generated as console output. This output will be submitted to the Evidence Store of the EMERALD framework in the format of the CertGraph ontology. D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 52 of 73 www.emerald-he.eu Figure 14. eknows sequence diagram 4.2.2 TWS – Trustworthiness System Component Name Trustworthiness System (TWS) Main functionalities The component provides the following functionalities: • Maintains an improved audit trail of evidence and assessment results. • Provides a manual and automatic way of verification of evidence and assessment results integrity. • Provides a record of information on a verifiable way (verification). • Provides a record of information on a permanent way (traceability). • Guarantees resistance to modification of stored data (integrity). Sub-components Description Blockchain network, use of a real implementation of a Blockchain network. EBSI will be considered as the first option for the deployment. Blockchain client, for providing the information (evidence/assessment results) to be saved on the Blockchain. Smart contract, deployed on the Blockchain network, for information (evidence/assessment results) writing and reading operations as well as events generation indicating the provision of new information. Viewer tool, for subscription to the Blockchain based events and notification to the different viewer clients. Graphical viewer client, for gathering and showing all the information saved on the Blockchain (and be able to manually verify it, without needing any interaction with the Blockchain). Automatic verification service, for evidence and assessment results integrity automatic check to be integrated in the GUI. D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 53 of 73 www.emerald-he.eu Main logical Interfaces offered Interface name Description Interface technology Blockchain client It provides: i) the required evidence and assessment results to be saved on the Blockchain, and ii) a way to obtain or check the evidence and assessment results saved on the Blockchain. REST API Graphical Viewer Client It provides a GUI to manually check evidence and assessment results saved on the Blockchain. Web Automatic Verification Service It provides a GUI for automatic verification of the integrity of evidence and assessment results. REST API Interaction with other components Interfacing Component Interface Description Assessment The Assessment will provide (and check, if needed) the information (evidence/assessment results) to be saved on the Blockchain by means of the Blockchain client interface. EmeraldUI The automatic verification service will provide the integrity verification information to the EmeraldUI to be shown to the EMERALD users. Auditors The auditors will check the information saved on the Blockchain by means of the graphical viewer client interface (manual way) or the automatic verification service interface (automatic way). Relevant sequence diagram/s See section 4.2.2.1 Requirements Mapping • TWS.01: Provide integrity proof of evidence • TWS.02: Provide integrity proof of assessment results • TWS.03: Provide access through REST API or graphical interface • TWS.04: Use a general purpose public-private Blockchain network Technology used Solidity, NodeJS, React, EBSI Related KR KR7: INTEROP – Interoperable assessment, evidence and catalogue data WP and task WP3 – T3.5 License Proprietary Partner TECNALIA D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 54 of 73 www.emerald-he.eu 4.2.2.1 Sequence diagram 4.2.2.1.1 System recording Figure 15 shows the sequence diagram of the TWS Recording component. TWS Recording receives from the Assessment component the information related to evidence and assessment results to be recorded in the Blockchain. Once this is done, the automatic verification service will be able to validate its integrity. Figure 15. TWS System Recording sequence diagram 4.2.2.1.2 System Verification Figure 16 shows the sequence diagram of the TWS Verification component. Supposing that in a previous step TWS Recording has recorded evidence in the Blockchain, an Auditor could want to check their integrity. For that, it uses the User Interface component, EmeraldUI, that calls the TWS Verification API. When required, the TWS Verification requests the current values of evidence stored in the Assessment component - the EMERALD’s internal evidence storage-, calculates the hash and compares it with the hash of the same evidence previously recorded in the Blockchain. The validation result can be true or false. The same process that happens for the evidence can be replicated for the assessment results. In the case of the automatic verification, it is not the Auditor user, through EmeraldUI, who calls the required components, retrieves hashes and makes the manual checking. In this case it only calls the TWS Verification, which includes a sub-component that executes the required process to retrieve the actual evidence -from the Assessment -, calculate its hash, and compare it with the stored evidence hash. The same automatic check process is replicated for the assessment results. D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 55 of 73 www.emerald-he.eu Figure 16. TWS System Verification sequence diagram 4.2.3 MARI - Mapping Assistant for Regulations with Intelligence Component Name Mapping Assistant for Regulations with Intelligence (MARI) Main functionalities The component creates an automatic association between: ● A security control and a security metric. ● Two security controls from two different certification schemes. D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 56 of 73 www.emerald-he.eu Subcomponents Des cription ● Feature extractor, based on a state-of-the-art NLP pre-trained model for transforming textual descriptions of metrics and controls into feature vectors. ● Clustering tool, for obtaining metric-control associations. Main logical Interfaces offered Interface name Description Interface technology API API to access MARI functionalities REST API Interaction with other components ● Repository of Controls and Metrics (RCM): MARI reads controls and metrics from the RCM and produces associations, which are then stored back in the RCM. ● EMERALD UI: MARI will interface with the EMERALD UI developed in WP4, through which it will be possible to view the results of control/metric associations and control/control associations. Relevant sequence diagram/s See section 4.2.3.1 Requirements Mapping ● MARI.01: AI-based ● MARI.02: Automatic association ● MARI.03: Performance Evaluation ● MARI.04: Usage and Visualization ● MARI.05: Strategies Technology used Python Related KR KR3_OPTIMA WP and task WP3 – T3.3 License Open Source with license Apache 2.0 Partner CNR 4.2.3.1 Sequence diagram Figure 17 shows the sequence diagram of the MARI component. MARI is an intelligent system capable of selecting the optimal set of metrics to evaluate the cloud system’s compliance within the certification schemes. The Compliance Manager triggers MARI, that will call the RCM to obtain the controls and metrics stored there. After the analysis, MARI will return the control/control associations and the control/metric associations to the RCM. D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 57 of 73 www.emerald-he.eu Figure 17. MARI sequence diagram 4.2.4 RCM - Repository of Controls and Metrics Component Name Repository of Controls and Metrics (RCM) Main functionalities The component provides the following functionalities: • Stores and manages certification schemes, supporting multi-scheme and multi-level certification. The RCM also incorporates the definition of the metrics used in EMERALD to assess evidence. • The RCM provides mechanisms to update the catalogues and maintain a versioning system and will allow importing and exporting catalogues into/from the RCM using OSCAL as exchange format. • Manages other related information, such as the controls mappings provided by the MARI component, the control implementation guidelines and a self-assessment questionnaire to assess compliance with a scheme. Subcomponents Description Frontend: This sub-component contains the graphical user interface of the RCM (It will be part of the EmeraldUI component and communicate with the backend via the API). It allows users to filter the view and select the set of information they want to check from the existing schemes (e.g., controls of a certain scheme, requirements of a certain assurance level, metrics related to some controls, etc). Backend: is the core sub-component of the RCM. It implements the APIs to perform the actual management of the scheme data, considering the filters set by the user through the UI or by calling the API. The RCM will contain two backends: i) Backend converter, which is dedicated to the scheme conversions to/from OSCAL, and ii) Backend, which deals with the management of schemes and metrics. D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 64 of 73 www.emerald-he.eu Figure 20. Evidence Store sequence diagram 4.2.7 Assessment Component Name Assessment Main functionalities The component provides the following functionalities: • Assesses evidence based on predefined metrics that are stored in the Repository of Controls and Metrics. Subcomponents Description Currently no division in subcomponents planned Main logical Interfaces offered Interface name Description Interface technology CLI A CLI is available Cobra26/Viper27 REST API/ gRPC API The following endpoints are available: • AssessEvidence to assess one evidence. • AssessEvidences to assess a stream of evidence. All endpoints are available via the REST API and gRPC API. 26 https://github.com/spf13/cobra 27 https://github.com/spf13/viper D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 65 of 73 www.emerald-he.eu Interaction with other components • Evidence Store: The Assessment retrieves evidence from the Evidence Store. • Orchestrator: • Registers the Assessment component in the Orchestrator (not yet implemented, to be discussed). • The Assessment sends the assessment results to the Orchestrator for storage. • The Assessment retrieves the metrics for the assessment from the Orchestrator. • Trustworthiness System: The Assessment component sends evidence and assessment results to the Trustworthiness System. Relevant sequence diagram/s See Section 4.2.7.1 Requirements Mapping List of requirements covered by this component: • ASSESS.01: Assessment based on evidence • ASSESS.02: Assessment rules for 80% of the defined metrics • ASSESS.03: Display cause of assessment result Technology used Go28, gRPC (using protobuf)29, Rego (Open Policy Agent)30 Related KR KR4_MULTICERT KR6_EMERALD UI/UX WP and task WP3 – T3.4 License Apache-2.0 Partner Fraunhofer AISEC 4.2.7.1 Sequence diagram Figure 21 shows the sequence diagram of the Assessment component. The Assessment component is responsible for assessing evidence based on predefined metrics. The calculated assessment results are eventually used by the Clouditor-Evaluation component to determine compliance with the relevant controls. At an initial registration phase, the Assessment component coordinates with the Orchestrator to receive instructions. The Assessment retrieves evidence from the Evidence Store to perform assessments. The result of the assessment is sent to the Orchestrator for storage. The Assessment interacts with the TWS to provide assessment results as well as the respective evidence. 28 https://go.dev/ 29 https://grpc.io/ 30 https://www.openpolicyagent.org/docs/latest/policy-language/ D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 66 of 73 www.emerald-he.eu Figure 21. Assessment sequence diagram 4.2.8 Evaluation Component Name Evaluation Main functionalities The component provides the following functionalities: • Aggregates assessment results assed by the Assessment component and determines the overall compliance status for a given control. • Evaluates the compliance of cloud services against controls and requirements of security catalogues. Subcomponents Description Currently no division in subcomponents planned Main logical Interfaces offered Interface name Description Interface technology CLI A CLI is available Cobra31/Viper32 REST API/gRPC API The following endpoints are available: • StartEvaluation starts the evaluation. • ListEvaluationResults lists stored evaluation results. All endpoints are available via the REST API and gRPC API. 31 https://github.com/spf13/cobra 32 https://github.com/spf13/viper D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 67 of 73 www.emerald-he.eu Interaction with other components • Orchestrator • Registers the Evaluation component in the Orchestrator (not yet implemented). • The Evaluation component retrieves assessment results from the Orchestrator. • Sends the evaluation results to the Orchestrator for storage. • Fetches controls from the Orchestrator. Relevant sequence diagram/s See Section 4.2.8.1 Requirements Mapping List of requirements covered by this component: • EVAL.01: Display cause of evaluation result • EVAL.02: Evaluation based on assessment results Technology used Go33, gRPC34 Related KR KR4_MULTICERT KR6_EMERALD UI/UX WP and task WP3 – T3.4 License Apache-2.0 Partner Fraunhofer AISEC 4.2.8.1 Sequence diagram Figure 22 shows the sequence diagram of the Evaluation component. The Evaluation component is responsible for aggregating and interpreting assessment results to determine overall compliance status of cloud services for a given control of a security catalogue. The Evaluation first registers itself into the Orchestrator. The Evaluation component obtains assessment results from the Orchestrator, processes them and determines the compliance status based on the mapping of metrics to controls of a security catalogue. The evaluation result is sent back to the Orchestrator for storage. 33 https://go.dev/ 34 https://grpc.io/ D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 68 of 73 www.emerald-he.eu Figure 22. Evaluation sequence diagram D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 69 of 73 www.emerald-he.eu 5 Conclusions This document is dedicated to introducing the EMERALD architecture to the reader. An overview of the system, the decomposition of EMERALD in 12 components, the information flow among them and a detailed view of them have been provided. These components will be in the future instantiated in the pilots defined in WP5. To complement the architecture, the general data model of the EMERALD framework, defined in D1.1 [1], has been presented. A Glossary is also included, with definition and examples of crucial terms. Following a multiple-perspective process, the requirements for the EMERALD framework have been designed. This document focuses on technical requirements, but we also included the Business requirement list, developed in WP5, and the UX/UI requirements, developed in WP4, for completion and analysis. A total of 44 functional requirements have been elicited, grouped in the 12 components that form the framework. These functional requirements are accompanied by 8 non-functional requirements, which are mostly system constrains or properties more than related to a particular component, so no effort has been spent in linking them to specific components. For each NFR, some hints on how we plan to fulfil them have been presented. An analysis of the requirements has been provided, where several matrices trace the coverage provided by the requirements to validate the pilots, the Key Results (KRs) or the Key Performance Indicators (KPIs). Also, the requirements prioritization and status at this V1 version of the EMERALD components in M12 is analysed. As a result, we have demonstrated that most of the Business requirements are covered by one or more technical requirements. That means that the corresponding component design is aligned with the final user’s view. Finally, we have provided a detailed view of the EMERALD framework, describing each component based on the component cards, which included sequence diagram developed with PlantUML to show their dynamic behaviour and interaction with other components. The future version of this document (D1.4 [2]) will review these requirements, their status and mappings, and could include new requirements as a result of the evolution of components, or of task related to the technical and pilots’ validation activities. D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 70 of 73 www.emerald-he.eu 6 References [1] EMERALD Consortium, “D1.1 Data modelling and interaction mechanisms-v1,” 2024. [2] EMERALD Consortium, “D1.4 EMERALD solution architecture-v2,” 2025. [3] EMERALD Consortium, “D1.7 EMERALD Integrated solution–v1,” 2025. [4] EMERALD Consortium, “D4.1 Results of the UI-UX requirements analysis and the work processes–v1,” 2024. [5] EMERALD Consortium, “D4.3 - User interaction and user experience,” 2024. [6] European Comission, “Regulation (EU) 2019/881 of the European Parliament and of the Council of 17 April 2019 on ENISA (the European Union Agency for Cybersecurity) and on information and communications technology cybersecurity certification and repealing Regulation (EU) No 52,” 10 2024. [Online]. Available: https://eurlex.europa.eu/eli/reg/2019/881/oj. [Accessed 10 2024]. [7] ISO, “ISO 9000:2015(en), Quality management systems — Fundamentals and vocabulary,” https://www.iso.org/obp/ui#iso:std:iso:9000:ed-4:v1:en, 2015. [8] “ISO/IEC 17788:2014 - Information technology — Cloud computing — Overview and vocabulary,” 2014. [9] National Institute of Standards and Technology (NIST), “SECURITY AND PRIVACY CONTROLS FOR INFORMATION SYSTEMS AND ORGANIZATIONS,” 2020. [10] ISO, “ISO/IEC 27000:2018 Information technology — Security techniques — Information security management systems — Overview and vocabulary,” 2018. [11] NIST - National Institute of Standards and Technology, “Key Concepts and Terms Used in OSCAL,” 10 2024. [Online]. Available: https://pages.nist.gov/OSCAL/resources/concepts/terminology/. [Accessed 10 2024]. [12] NIST - National Institute of Standards and Technology, “Cloud Computing Service Metrics Description,” 24 April 2018. [Online]. Available: https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.500-307.pdf. [Accessed 10 2024]. [13] EMERALD Consortium, “D2.2 - Source Evidence Extractor – v1,” 2024. [14] EMERALD Consortium, “D2.4 - AMOE – v1,” 2024. [15] EMERALD Consortium, “D2.6 - ML model certification – v1,” 2024. [16] EMERALD Consortium, “D2.8 Runtime evidence extractor – v1,” 2024. [17] EMERALD Consortium, “D3.1 Evidence assessment and Certification–Concepts-v1,” 2024. D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 71 of 73 www.emerald-he.eu [18] EMERALD Consortium, “D5.1 Pilot definition, set-up & validation plan,” 2024. [19] EMERALD Consortium, “EMERALD - Annex 1 - Description of Action - GA 101120688,” 2022. D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 72 of 73 www.emerald-he.eu APPENDIX A: Current status of requirements Table 14 depicts the status of the technical requirements, ordered by component. The peachcoloured lines highlight those requirements that are foreseen for M12. The “Timeline” column states the month foreseen to complete the implementation, and the associated Milestone (see the codes below) in an abbreviated form, where “C” stands for the Components version, and “I” stands for Integration version. For example: - C-v1 = MS2: Components V1 (M12) - I-v3 = MS8: Integrated audit suite V3 (M34) Table 14. Status of the Technical requirements Req. ID Title Priority Timeline Status AI-SEC.01 The extractor tool includes selected criteria MUST M12 (C-v1) 35% AMOE.01 Upload PDF document MUST M12 (C-v1) 90% AMOE.02 Provision of extracted evidence to EvidenceStore (Orchestrator/Clouditor) MUST M24 (C-V2) 50% AMOE.03 Refine evidence extraction approach MUST M24 (C-V2) 0% AMOE.04 Compare results from multiple documents SHOULD M12 (C-v1) 70% AMOE.05 Select metrics per document SHOULD M24 (C-V2) 0% AMOE.06 Classify document, select respective metrics (optional) MUST M34 (I-v3) 0% AMOE.07 Metric states SHOULD M24 (C-V2) 0% CLDISC.01 Discovery of security properties of infrastructure components MUST M30 (I-v2) 40% CODYZE.01 Extraction of security features from source code MUST M30 (I-v2) 20% EKNOWS.01 Integration into existing systems MUST M18 (I-v1) 30% EKNOWS.02 Resilience while analysing erroneous code SHOULD M24 (C-V2) 70% EKNOWS.03 Multi-language support MUST M24 (C-V2) 50% EKNOWS.04 Support EMERALD evidence format MUST M18 (I-v1) 0% EKNOWS.05 Static code analysis MUST M24 (C-V2) 60% TWS.01 Provide integrity proof of evidence MUST M12 (C-v1) 75% TWS.02 Provide integrity proof of assessment results MUST M12 (C-v1) 75% TWS.03 Provide access through REST API or graphical interface MUST M24 (C-V2) 50% TWS.04 Use a general purpose public-private Blockchain network MUST M24 (C-V2) 5% MARI 1.0 AI-based MUST M30 (I-v2) 15% MARI 2.0 Automatic association MUST M30 (I-v2) 15% MARI 3.0 Performance evaluation MUST M30 (I-v2) 15% MARI 4.0 Usage and visualization MUST M30 (I-v2) 15% D1.3 – EMERALD solution architecture-v1 Version 1.0 – Final. Date: 31.10.2024 © EMERALD Consortium Contract No. GA 101120688 Page 73 of 73 www.emerald-he.eu Req. ID Title Priority Timeline Status MARI 5.0 Strategies MUST M30 (I-v2) 15% RCM.01 Multi-schema support MUST M12 (C-v1) 90% RCM.02 Accessible by the rest of components MUST M12 (C-v1) 100% RCM.03 Include metrics for all schemas supported MUST M12 (C-v1) 30% RCM04 Mapping of schemes SHOULD M30 (I-v2) 10% RCM.05 Import/export of security schemes in OSCAL MUST M30 (I-v2) 40% RCM.06 Import/export of security schemes in CSV format COULD M12 (C-v1) 60% RCM.07 Support for personalized catalogues MUST M30 (I-v2) 0% RCM.08 Support updating/versioning of schemes SHOULD M30 (I-v2) 10% ORCH.01 Final certificate decision MUST M24 (C-v2) 0% ORCH.02 REST API Gateway for UI MUST M12 (C-v1) 15% ORCH.03 Role Based Access Control MUST M24 (C-v2) 25% ORCH.04 Manage Tools (such as Evidence Extractors) via API MUST M18 (I-v1) 0% ORCH.05 IssueORCH.05 Provide an API for audit workflow MUST M30 (I-v2) 0% ESTORE.01 Storage of ontology entities in graph database MUST M18 (I-v1) 15% ESTORE.02 Allow Interaction with Third-Party Evidence Collectors SHOULD M34 (I-v3) 15% ASSESS.01 Assessment based on evidence MUST M30 (I-v2) 15% ASSESS.02 Assessment rules for 80% of the defined metrics MUST M30 (I-v2) 15% ASSESS.03 Display cause of assessment result COULD M30 (I-v2) 0% EVAL.01 Display cause of failing evaluation result COULD M30 (I-v2) 0% EVAL.02 Evaluation based on assessment results MUST M30 (I-v2) 15% The list of Milestones of the EMERALD project are [19]: • MS1: Project baselines and definition (M9) • MS2: Components V1 (M12) • MS3: Integrated audit suite V1 (M18) • MS4: Pilots V1 (M20) • MS5: Components V2 (M24) • MS6: Integrated audit suite V2 (M30) • MS7: Pilots V2 (M32) • MS8: Integrated audit suite V3 (M34) • MS9: Final evaluation report and impact analysis (M36)