D2.1: Use Case Specification (first version)
Abstract
This deliverable presents the outcomes of Task 2.1 – Pilots Use Cases Specifications of Work Package 2 – Use Cases Specifications, Requirements and Architecture of RESCALE during its first iteration. It includes the initial description and specification of the two pilot use cases. It provides a static and dynamic overview of the pilots’ components and presents the two involved SMEs. The deliverable also describes the fundamental problems that the pilots face and will solve through the RESCALE project.
Full text
D2.1: Use Case Specifications (First Version) PROJECT Project Number 101120962 Project Acronym RESCALE Project Title Revolutionised Enhanced Supply Chain Automation with Limited Threats Exposure Start Date 01.10.2023 Programme HORIZON-CL3-2022-CS-01-02 DELIVERABLE Deliverable Type Report Workpackage WP2 Deliverable Lead CC Editors Marton Sipos Contributors PST Dissemination Level Public Abstract This deliverable presents the outcomes of Task 2.1 – Pilots Use Cases Specifications of Work Package 2 – Use Cases Specifications, Requirements and Architecture of RESCALE during its first iteration. It includes the initial description and specification of the two pilot use cases. It provides a static and dynamic overview of the pilots’ components and presents the two involved SMEs. The deliverable also describes the fundamental problems that the pilots face and will solve through the RESCALE project. Disclaimer The information in this document is provided “as is”, and no guarantee or warranty is given that the information is fit for any particular purpose. The content of this document reflects only the author’s view – the European Commission is not responsible for any use that may be made of the information it contains. The users use the information at their sole risk and liability. This project has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No 101120962
D2.1: Use Case Specifications (First Version) Document Revisions & Quality Assurance Internal Reviewers 1. Danijela Boberic Krsticev, UNSPMF 2. Gogos Anastasios, INTRA Revisions Version Date Partner Overview 1.0 29/02/2024 CC & PST Final version 0.9 28/02/2024 UNSPMF & INTRA Reviewer comments 0.8 15/02/2024 PST Updated Pilot1 based on partner feedback 0.7 09/02/2024 CC Updated Pilot2 based on partner feedback 0.6 28/01/2024 PST Added figure and description to Pilot1 0.5 19/01/2024 CC Content added to sections 1 and 4 0.4 16/01/2024 PST Draft description of Pilot1 0.3 16/01/2024 CC Updated description of Pilot2 0.2 08/01/2024 CC Draft description of Pilot2 0.1 04/01/2024 CC Table of Contents and instructions RESCALE – Public – Page 2 / 31
Table of Contents 1 Introduction 7 1.1 PurposeofthisDocument............................. 7 1.2 Methodology ................................... 7 1.3 DocumentStructure................................ 9 1.4 IntendedAudience ................................ 9 2 Pilot 1: Auto-Placement Security Aware Augmented Data-Flow and Infrastructure 10 2.1 PeerStritzingerGmbH .............................. 10 2.2 Overview ..................................... 10 2.3 SystemArchitecture................................ 11 2.4 Workflows..................................... 13 2.5 Infrastructure and Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.6 ProblemStatement ................................ 15 2.7 Threatmodel ................................... 16 2.8 Issues with Existing Approaches . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.9 Impact of RESCALE on the Pilot Use Case . . . . . . . . . . . . . . . . . . . 18 2.10Successcriteria .................................. 19 3 Pilot 2: Privacy-by-Design Distributed Cloud and Edge Storage 20 3.1 ChocolateCloudApS............................... 20 3.2 Overview ..................................... 20 3.3 SystemArchitecture................................ 21 3.4 Workflows..................................... 23 3.5 Infrastructure and Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.6 ProblemStatement ................................ 25 3.7 Threatmodel ................................... 26 3.8 Issues with Existing Approaches . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.9 Impact of RESCALE on the Pilot Use Case . . . . . . . . . . . . . . . . . . . 28 3.10Successcriteria .................................. 28 4 Conclusions and Future Plans 29 3
List of Figures 1 Automated Data Flow Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2 Overview of different services and solutions developed by Chocolate Cloud as part of the SkyFlok product family. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3 Overview of different all SkyFlok components, their interfaces and interactions. . . 22 4 Overview of SkyFlok file storage and retrieval. . . . . . . . . . . . . . . . . . . . 24 5 The steps required to download a file from SkyFlok using a pre–signed URL. . . . 25 4
D2.1: Use Case Specifications (First Version) Abbreviations Abbreviation Full form 2FA Two–factor Authentication API Application Programming Interface APT Advanced Persistent Threat CC Chocolate Cloud ApS CI/CD Continuous integration/Continuous Deployment DDS Data Distribution Service DSCG Device Supply Chain Graph GCP Google Cloud Platform GDPR General Data Protection Regulation IoT Internet of Things MIT Massachusetts Institute of Technology OPC-UA Open Platform Communications - Unified Architecture PST Peer Stritzinger GmbH REST Representational State Transfer RLNC Random Linear Network Coding SaaS Software as a Service SME Small and Medium-sized Enterprises SSCG Software Supply Chain Graph TBOM Trusted Bill of Material TSN Time-Sensitive Networking TrustOR Trust Orchestrator Framework URL Uniform Resource Locator VM Virtual Machine RESCALE – Public – Page 5 / 31
D2.1: Use Case Specifications (First Version) Executive Summary This deliverable presents the two pilot use cases as part of the first iteration of Task 2.1. It includes their descriptions and their initially collected requirements. It focuses on defining the main problem the uses cases seek to solve as part of the project, what is missing from current state of the art solutions and how RESCALE will solve the main problem. The document is divided into five main sections beyond this executive summary. Section 1 serves as an introduction to the document. It provides information about the document’s purpose, including its role in the RESCALE project, the methodology with which its contents were created, its structure and the audience to which it is mainly addressed. Section 2 is dedicated to RESCALE’s first pilot use case: Auto-Placement Security Aware Augmented Data-Flow and Infrastructure. It describes the use case provider, the pilot’s architecture, infrastructure and workflows. It defines the core problems that will be addressed in RESCALE, highlighting any issues with existing approaches. It presents an initial threat model as well as a set of success criteria to serve during the evaluation of the pilot use case. Section 3 is dedicated to RESCALE’s second pilot use case: Privacy-by-Design Distributed Cloud and Edge Storage and matches the structure of the previous section. Section 4 summarizes the contents of the deliverable and provides a brief description of its role in the project’s life–cycle. RESCALE – Public – Page 6 / 31
D2.1: Use Case Specifications (First Version) 1 Introduction 1.1 Purpose of this Document This deliverable presents the first outcomes of the Task 2.1 - Pilots Use Cases Specifications, during its first iteration. The project adopts a two-cycle approach regarding this task. In the first cycle, the pilots are specified according to what has been described in the proposal including any additional updates that may be required. In the second cycle, the specifications will be refined, taking into consideration all the work that has been done during the first cycle implementations. Task 2.1 involves the description of the full specification of the RESCALE pilot use cases. Within this task, the partners providing the pilots (Peer Stritzinger GmbH – PST and Chocolate Cloud ApS – CC), in collaboration with those developing RESCALE’s technologies, will refine them to the level of detail required so that their key functional aspects are completely specified. The specifications provided in this deliverable are an important part of the project’s first cycle. They will be used in Task 2.2 in order to extract functional and non-functional user requirements, forming the starting point of Deliverable 2.3 - Technology Analysis and Requirements Specification (first version), due in M6. Additionally, in the second cycle, all partners will work together to design and specify the required trial procedures and usage scenarios of these applications. These will drive later efforts to define testing, validation, demonstration and evaluation procedures as part of Work Package 5. The specifications will also be updated. The results will be reported in an updated version of this document entitled Deliverable 2.2 - Use Case Specification (final version), provided in M19 of the project timeline. It will feature updates based on the results of the first stage of development and evaluation of the project. 1.2 Methodology The pilot use cases play a crucial role in establishing many of the goals and requirements of RESCALE. As such, it is important that their own requirements and specifications are collected early in the lifecycle of the project. The partners leading the use cases (use case providers from now on), PST and CC, were provided with the below list of instructions, breaking down the task into smaller steps. Context was given as to how the different parts would be used in the later stages of the project. The project’s proposal was used as a starting point. •Overview: Provide a high-level overview of the pilot use-case, ideally with figures. The goal of this subsection is to gently introduce the reader to the pilot use case and set the stage for the subsequent subsections. •System architecture: Describe the system architecture, including all software and hardware components that will be affected by the solutions developed in RESCALE. Describe the software and hardware supply chain of the Pilot system. The items listed in this section will be the ones that undergo static and dynamic testing by the RESCALE platform RESCALE – Public – Page 7 / 31
D2.1: Use Case Specifications (First Version) tools and will have an associated entry in the SSCG and/or DSCG parts of the TBOM. •Workflows: Provide a high-level description of one or more relevant workflows that will be examined in RESCALE. Show how the software and hardware components interact with each other. This section will be used to establish the producer-consumer graph of components in the TBOM. •Infrastructure and deployment: Describe the infrastructure and deployment of the pilot application. Detail level should be determined by how relevant these aspects are to RESCALE. •Problem definition: Define the core problem to be solved in RESCALE. The identified issues should more or less match those we will likely tackle in the project. Provide this input based on the discussions in the WP telcos. This part should be kept relatively highlevel, as the formalized user requirements are specified separately in Deliverable 2.3. •Threat model: Describe the threat model to be considered during the project. Consider issues relevant to the project’s envisioned outcomes. •Issues with existing solutions: Describe the current solutions of ensuring security of software and hardware components, including 3rd party tools in the supply chain (both open-source and proprietary). These solutions will be augmented or replaced by new or enhanced solutions provided by the RESCALE platform. Point out the shortcomings of existing security-related techniques as well as missing security solutions and their justification (e.g., why would they be beneficial). Focus on the missing methods that are targeted in RESCALE. The RESCALE platform will provide solutions that fix the current issues and fill in the holes identified here. •Impact of RESCALE on the pilot use case: Provide initial ideas of how the RESCALE platform components will interact with or get integrated into the pilot’s systems to carry out the required operations. Refer to the same concepts and components described in the System Architecture subsection. This text will guide the RESCALE platform’s system design, specifically its interfaces and integrations with the pilot systems. •Success criteria: Define a list of success criteria that can be used to assess whether the RESCALE platform has met the listed requirements. Be consistent with the KPIs defined in the DoA. This list, as well as the actual contents of the deliverables, were written after a series of regular discussions on the topic. As a preparatory step, the pilot use case providers were asked to provide a tentative list of high-level goals they wished to achieve in the project. They presented these goals to the technical partners to understand whether they fit within the scope of the project, their complexity and the effort required to implement a solution for them. Following this, the use case providers provided the specifications of their pilot applications based on the previously described instructions. As part of Task 2.2, the discussions extended to the user requirements of the two pilots. Coinciding with the preparation of this deliverable, an initial list of functional and non-functional requirements was defined to be included in Deliverable 2.3. RESCALE – Public – Page 8 / 31
D2.1: Use Case Specifications (First Version) 1.3 Document Structure The present deliverable is split into the following major chapters: •Introduction: describes the purpose and structure of this deliverable and its role in the project. •Pilot 1: Auto-Placement Security Aware Augmented Data-Flow and Infrastructure: presents the first pilot use case through the schema described in Section 1.2. •Pilot 2: Privacy-by-Design Distributed Cloud and Edge Storage: presents the second pilot use case through the schema described in Section 1.2. •Conclusions and Future Plans: concludes the document, pointing to how it will be used in the continuation of the project. Finally, a list of sources that have been referred to in the various sections of the document is included. 1.4 Intended Audience This document is publicly available and should be of use to anyone interested in the detailed description and specification of the two pilot use cases of the RESCALE project. RESCALE – Public – Page 9 / 31
D2.1: Use Case Specifications (First Version) 2.7 Threat model The threat model for the pilot use-case of Peer Stritzinger GmbH (PST) within the RESCALE project encompasses various potential security risks and vulnerabilities that may impact the distributed computing environment. This model serves as a foundation for developing robust security strategies and countermeasures. External Attacks: The system faces threats from external attackers who may attempt to gain unauthorized access, disrupt services, or compromise data integrity. This includes risks such as network intrusions, denial of service attacks, and malware. • Attacks through SaaS website, privilege elevation, session hijacking etc. • Information leakage between VMs on the same machine via transient execution attacks like Meltdown or Spectre. Especially every component that holds key material. • Side channel attacks via power or timing analysis or similar. Timing analysis is possible remotely and locally whereas the other attacks in this family require physical access. With IoT devices it’s not uncommon that they operate in hostile environments, which can provide this kind of physical access. Internal Threats: Internal threats include risks from within the organization or network, such as unauthorized access or misuse of data by insiders. These threats may be intentional or due to negligence. • Users of the IoT platform can apply all external attacks • Developers of software components on the IoT platform that are provided to other users can steal, corrupt and delete data of these users Data Interception and Manipulation: Given the distributed nature of the network, data transmitted over various channels is susceptible to interception and manipulation. Securing data in transit and at rest is a critical aspect of the threat model. Hardware and Software Vulnerabilities: Vulnerabilities in hardware components and software applications pose significant risks. These vulnerabilities can be exploited to gain unauthorized access or disrupt system operations. Software vulnerabilities in the supply chain have more and more impact to overall system security. The threat from the supply chain also extends from accidental vulnerabilities to intentional ones which can be harder to detect. Compliance and Regulatory Risks: Non-compliance with industry standards and regulations, especially in terms of data protection and privacy, is another area of concern. Ensuring adherence to these standards is vital for the integrity of the system. PST’s approach within RESCALE is to address these threats through a combination of advanced security technologies, rigorous testing and validation processes, and continuous monitoring and updating of the system. This proactive stance ensures that the infrastructure remains resilient against evolving security challenges. RESCALE – Public – Page 16 / 31
D2.1: Use Case Specifications (First Version) 2.8 Issues with Existing Approaches Peer Stritzinger GmbH (PST) currently employs a variety of security solutions to safeguard its software and hardware components, including the integration of third-party tools in the supply chain. While these existing approaches provide a certain level of security, there are notable shortcomings that the RESCALE platform aims to address and improve upon. Current Security Solutions: •Standard Security Protocols: PST utilizes industry-standard security protocols for data encryption and secure communication. While effective to an extent, these protocols often do not account for more sophisticated or novel attack vectors. •Third-Party Security Tools: The integration of both open-source and proprietary security tools is part of PST’s approach. These tools offer functionalities like vulnerability scanning, intrusion detection, and code analysis. However, their effectiveness can be limited by the lack of comprehensive coverage and integration. •Manual Security Practices: Current practices often involve manual processes for security checks and updates, which can be time-consuming and error-prone, leading to potential vulnerabilities. Shortcomings and Missing Solutions: The existing security measures, while foundational, present several limitations: •Lack of End-to-End Security Integration: There is a need for a more integrated security approach that covers the entire spectrum of the distributed system, from hardware components to application-level software. •Inadequate Response on Deployed Systems: The ability to respond at runtime to security threats is often limited, making the system vulnerable to rapidly evolving attacks. •Insufficient Adaptability to New Threats: Existing tools may not be well-equipped to adapt to new or emerging security threats, especially in the context of advanced persistent threats (APTs) and zero-day exploits. •Dependency on Third-Party Tools: Relying heavily on third-party tools, especially proprietary ones, can create dependencies and potential security gaps due to lack of control and customization. Justification for Enhanced Solutions: Enhancing or replacing current solutions with those provided by the RESCALE platform is justified by: •Need for Comprehensive Security Framework: A holistic security framework, capable of addressing the unique challenges of distributed systems, is essential for ensuring end-toend security. This includes a particular emphasis on the supply chain, ensuring that every component, from software development to deployment, is secure and trusted. RESCALE – Public – Page 17 / 31
D2.1: Use Case Specifications (First Version) •Requirement for Automated and Integrated Tools: Automated security tools that are wellintegrated into the system architecture can significantly reduce response times and human errors, enhancing the overall security level. This integration is crucial not just within the system’s operational boundaries but extends to encompass the entire supply chain, from third-party software components to hardware procurement. •Supply Chain Security and Compliance: Ensuring the security of the supply chain is a critical component of modern cybersecurity strategies. The RESCALE platform aims to provide comprehensive tools for monitoring, analyzing, and securing the supply chain, thereby preventing the introduction of vulnerabilities through third-party components. 2.9 Impact of RESCALE on the Pilot Use Case The integration of the RESCALE platform within the pilot use-case of Peer Stritzinger GmbH (PST) is anticipated to have a significant and transformative impact. The RESCALE components are designed to interact synergistically with the existing system architecture of PST, enhancing its capabilities and addressing the current limitations. This section provides an initial outline of how RESCALE’s components will integrate into and interact with PST’s pilot systems. Integration with Computing Nodes: •Enhanced Security Protocols: RESCALE’s advanced security testing protocols will be integrated into PST’s computing nodes, including servers and edge devices, to provide stronger and more adaptable security vulnerability detection. •Dynamic Vulnerability Assessment: Continuous vulnerability assessment tools from RESCALE will be implemented to monitor and analyze deployed software, ensuring timely detection and response to potential threats. Software and Hardware Supply Chain Integration: •Supply Chain Security Platform: The integration of RESCALE’s supply chain security tools into the CI that builds the IoT platform will ensure that all software and hardware components in PST’s supply chain are secure. •Supply Chain Security Users: Integrating RESCALE’s supply chain security tools into the IoT platform itself enables PST’s customers to easily maintain the supply chain security of all their components as well as third party components offered on the platform app store. Enhancements in Embedded Systems and IoT Devices: •IoT Security: IoT and embedded systems in PST’s network will be equipped with RESCALE’s IoT-specific security solutions, providing robust protection against device-level threats. RESCALE – Public – Page 18 / 31
D2.1: Use Case Specifications (First Version) •Firmware Updates and Management: Firmware management tools from RESCALE will ensure that these devices are always running software scanned by RESCALE’s analysis tools. Automated Deployment and Monitoring: •CI/CD Integration: The integration of RESCALE’s tools into PST’s CI/CD pipeline will enhance the security and efficiency of the software development, testing, and deployment processes. •CI/CD Integration for Platform Users: The integration of RESCALE’s tools into PST’s platform integrated CI/CD will enhance the security and efficiency of the software development, testing, and deployment processes for the users of the platform. The impact of the RESCALE platform on PST’s pilot use-case is expected to be extensive, touching every aspect of the system architecture. The platform’s components are designed to seamlessly integrate with PST’s existing infrastructure, providing enhanced security, efficiency, and adaptability. This integration will not only address the current shortcomings but also prepare the system for future technological advancements and security challenges. 2.10 Success criteria The success criteria for the integration of the RESCALE platform into PST’s pilot use-case are anchored in the achievement of key operational and security enhancements, as outlined in the project’s requirements. Firstly, a significant reduction in the incidence and severity of security breaches or vulnerabilities within the system is a primary criterion. This includes the effective detection of threats by static and dynamic testing, demonstrating the robustness of RESCALE’s security protocols and tools. Secondly, the seamless integration of RESCALE components with PST’s existing system architecture without causing disruptions or performance degradation is crucial. This entails the smooth functioning of automated deployment processes, real-time monitoring systems, and the IoT security framework. Additionally, compliance with industry-standard security practices and regulations, as facilitated by RESCALE’s automated compliance tools, is a critical measure of success. Finally, the adaptability of the system to incorporate future security updates and technological advancements, maintaining its resilience against evolving cyber threats, will be a definitive indicator of the long-term success and sustainability of the RESCALE integration. RESCALE – Public – Page 19 / 31
D2.1: Use Case Specifications (First Version) 3 Pilot 2: Privacy-by-Design Distributed Cloud and Edge Storage 3.1 Chocolate Cloud ApS Chocolate Cloud ApS[5] (CC) is a high-tech start-up based in Denmark with deep MIT roots and with a global perspective. CC delivers software products for efficient and privacy-preserving Multi-Cloud data storage (hybrid clouds[6], multiple public clouds [7], multiple private clouds) as well as a blazing fast technology to reduce storage by a factor of 2 in OpenStack (Objectbased storage) and Hadoop (BigData). CC’s vision goes beyond the current Cloud storage market to create solutions that include edge resources and IoT storage devices. This is key in meeting the needs of new emerging lowlatency applications, such as those developed to run over 5G networks. To this end, CC has participated in several research projects [8,9,10] aimed at developing their unique privacy, security and performance capabilities at the edge of the network. In RESCALE, CC’s goal is twofold. First, CC aims to validate and improve the security of their existing products. Second, CC seeks to bring the technology they have developed in previous projects closer to a marketready state through the tools developed in the project. 3.2 Overview The second pilot use case centers around Chocolate Cloud’s distributed cloud and edge storage solutions shown on Figure 2. Data security and privacy are the key commitments CC makes to its users. All CC products share the same core ideas such as having different layers of encryption to protect data both in transit and at rest and using Random Linear Network Coding [11], an erasure code with privacy benefits. However, CC has yet to implement a comprehensive, holistic security assessment mechanism that covers its entire supply chain. This is an urgently needed development that will be achieved using the tools and techniques developed in RESCALE. Rather than creating novel, custom security mechanisms, the pilot use case will showcase the innovations of the project, relying on them to increase the security of CC’s products. SkyFlok SkyFlok[12] is Chocolate Cloud’s most important product aimed at end-users. It is a secure file storage solution designed to tailor to the needs of both businesses and individuals. It enables users to easily and reliably store their data in the cloud. Compared to competing solutions, it gives customers full control over where their data is stored by leveraging a multi-cloud approach. SkyFlok supports the three major cloud providers (Amazon, Google, Microsoft) as well as several smaller local solutions (OVH, Deutsche Telekom, Orange Business Cloud, IONOS, Exoscale, etc.). Through the use of EU-based providers, SkyFlok offers GDPR-compliant storage through 54 GDPR-compliant locations. This service has been commercially available since Q1 2018 and has had more than 750, mostly SME teams using it since Q1 2020. RESCALE – Public – Page 20 / 31
D2.1: Use Case Specifications (First Version) Figure 2: Overview of different services and solutions developed by Chocolate Cloud as part of the SkyFlok product family. SkyFlok for Enterprise Medium and large businesses (250+ employees) that are SkyFlok customers would like to extend their use of Chocolate Cloud’s SkyFlok service. At the moment, SkyFlok works great for file-based collaboration and file sharing workflows as well as for data archival purposes. However, given its fully cloud-based architecture, it lags behind in terms of latency compared to an on-premises storage solution. By introducing an On-premises Storage Gateway and moving data closer to the edge, download and upload latencies can be greatly improved. While many customers choose SkyFlok over competing solutions thanks to the privacy guarantees it offers, privacy concerns remain a major impediment to more wide-spread adoption of cloud storage in general. Due to legal requirements or internal policies, enterprises want strong guarantees that their data cannot be accessed by third parties, including the storage provider. Conventional cloud storage can only achieve this to a limited degree. Moving the file encryption/decryption process on premises, under full control of the enterprise is key to providing these guarantees. Chocolate Cloud has developed SkyFlok for Enterprise, a solution that extends SkyFlok by offering edge storage locations to customers who already own the required hardware or are willing to invest in on-premises storage infrastructure. These devices will show up in SkyFlok and will be used in combination with the traditional cloud-based storage services. SkyFlok S3 Additionally, Chocolate Cloud is preparing to launch SkyFlok S3, an object storage service with a public S3 API which requires some privacy-sensitive processes to be hosted at the Edge or in the Cloud. Just like SkyFlok for Enterprise, the S3 interface is provided by a set of S3 Gateways. However, instead of the clients’ own infrastructure, these are deployed by CC to public cloud infrastructure. Currently CC is utilizing Fly.io to deploy S3 Gateways in different locations across Europe in an effort to reduce file access latency for a geographically varied user base. 3.3 System Architecture An overview of the software components that make up the SkyFlok services can be seen on Figure 3. The most important dataflows and interfaces are also shown. RESCALE – Public – Page 21 / 31
D2.1: Use Case Specifications (First Version) Figure 3: Overview of different all SkyFlok components, their interfaces and interactions. Skyflok.com backend The Skyflok.com backend is a set of microservices that manage all aspects related to file storage and sharing, user and team management and several connected aspects. All microservices have been written in Python and store entities in either a traditional relational database or in Google Datastore. It is a core component, shared by SkyFlok, SkyFlok for Enterprise and SkyFlok S3 services. SkyFlok web application The SkyFlok web application (Webapp) is a Javascript and VueJS application that is the sole interface used by SkyFlok users. It follows a thin client design approach, communicating with the backend through a REST API. To ensure scalability, it also performs encryption, erasure coding and manages transfers to and from the cloud storage locations. RESCALE – Public – Page 22 / 31
D2.1: Use Case Specifications (First Version) On-premises Storage Gateway The On-premises Storage Gateway component is an extension to the original SkyFlok deployment, developed as part of the SkyFlok for Enterprise solution. It is a self-hosted Python application that provides an industry–standard S3 interface for object storage. It is stateless, beyond a configurable file and metadata cache and performs all encryption, erasure coding and data transfer tasks typically performed by the user’s browser. This component is not shown separately on Figure 3, but is very similar to the SkyFlok S3 Gateway in terms of its place in the system architecture. A key differentiator is its ability to also use SkyFlok edge devices as storage locations. SkyFlok edge devices To be able to offer a versatile edge storage solution for enterprises, CC has created a customized version of MinIO [13] that can be deployed by the customer onto existing infrastructure. This approach has been selected as it makes it possible to integrate a wide range of storage resources into SkyFlok for Enterprise. SkyFlok S3 Gateway The SkyFlok S3 Gateway is the equivalent of the On-premises Storage Gateway for the SkyFlok S3 service. It shares most of its features, but its implementation is significantly different as it has been designed to be deployed to a public cloud. The following list contains an overview of the components that should be checked using static code analysis as well as the programming language in which they were written: • SkyFlok backend microservices (Python), • SkyFlok S3 Gateway (Python) and On-premises Storage Gateway (Python), • SkyFlok.com webapp (Javascript, VueJS2), • SkyFlok Object Storage Developer Portal (Typescript, VueJS 3), • RLNC library used for erasure coding (C++). 3.4 Workflows Here we present the high-level overview of SkyFlok’s file upload and download workflows. These are provided as relevant examples, showcasing the interactions between many of the components of CC’s supply chain. A more complete and detailed description of workflows will be provided later within the project to help establish the producer-consumer graph of components in the TBOM. In SkyFlok, to achieve outstanding security and accessibility, files are encrypted and erasurecoded with added redundancy, and then the resulting fragments are distributed across multiple cloud storage providers and countries. SkyFlok runs these computations inside the client’s browser without requiring the backend to carry out most securityand privacy-sensitive steps (e.g., encryption, Random Linear Network Coding). In SkyFlok for Enterprises and SkyFlok RESCALE – Public – Page 23 / 31
D2.1: Use Case Specifications (First Version) S3, these steps are delegated to Gateways deployed either at the edge on the client’s private infrastructure or as a service running in a public cloud. In all three cases, the Skyflok.com backend as well as the storage locations exclusively interact with encrypted, erasure-coded data. The encryption keys are stored by the Skyflok.com backend. Figure 4: Overview of SkyFlok file storage and retrieval. Figure 4 shows a high-level view of how SkyFlok files are uploaded and retrieved. The erasure– coded fragments are distributed across the selected public cloud locations. The Skyflok.com backend coordinates the process, acting as a sort of conductor. File data is transferred directly between the browser and the cloud location. Retrieving files is quite similar and involves downloading enough erasure–coded fragments, then decoding and finally decrypting. The whole process is coordinated with the use of a storage policy. A storage policy is like a recipe in the sense that it defines what encryption and erasure coding scheme is used (as well as the schemes’ parameters) and what chosen locations for the coded fragments are selected. Each SkyFlok customer has their own custom storage policy defined, based on their particular requirements as well as on their physical location. To be able to connect a user directly with the fragments on the cloud and edge locations, pre– signed URLs are used. The links used to upload and download fragments are requested by the browser from the Skyflok.com backend. Because the links point to protected resources, they pose a security challenge. How can the browser be trusted with the credentials necessary to access these resources? The solution is to create a cryptographic signature and include it in the link. This creates a pre–signed URL which does not require additional credentials to access the protected resource. The URLs have a short lifespan and can generally only be used once. The signature is calculated using asymmetric-key cryptography and is guaranteed to be different every time. Figure 5 illustrates the steps taken when a file is downloaded using a pre–signed URL. RESCALE – Public – Page 24 / 31
D2.1: Use Case Specifications (First Version) Figure 5: The steps required to download a file from SkyFlok using a pre–signed URL. 3.5 Infrastructure and Deployment Infrastructure: One of the core challenges CC faces in attesting that its software is preserving the privacy and security of the data it stores for its customers comes from the lack of direct control over the infrastructure where the software is deployed to. As detailed in Section 3.3 and presented on Figure 3, the Skyflok.com backend is a set of microservices running on the Google Cloud Platform (GCP), using the Google App Engine environment. The On-premises Storage Gateway, as its name suggests, is deployed on a customer’s existing infrastructure and is monitored and managed by the customer, not CC. The SkyFlok S3 Gateway is currently deployed to Fly.io. Finally, SkyFlok users access the service using a web browser, a software component that is part of the chain, but also outside the supervision of CC. Deployment mechanisms: Beyond the lack of control over the infrastructure, the deployment process itself can potentially be a security vulnerability. Fortunately, both GCP and fly.io have a strong security model that requires any changes to deployed services be made by developers in the possessions of a private key associated with the company. However, this still requires trust in the provider and makes a security audit more difficult as the infrastructure upon which CC’s services are running is not publicly documented. 3.6 Problem Statement Fundamental problem: Privacy and security audits are crucial to gaining new customers and supporting new cloud storage providers in our ecosystem. Thus, automated and dynamic analysis mechanisms that can validate both CC’s own software and third-party software libraries and/or host systems are needed to scale the business. Furthermore, trusted third-party certifications for privacy and security can be eased and streamlined in the future, reducing operating costs. For example, one privacy certification may cost tens of thousands of Euros yearly aside from the internal costs of running the verification and certification process. Static code security testing: One of the key building blocks to solving the problem lies in static code security analysis techniques. However, in order for them to work, they must be able to handle the complexities of the supply chain, the large number of dependencies and the varied hardware infrastructure. Through a comprehensive suite of traditional static code analysis testing, a list of alerts should be generated. The list should be complete, highlighting potential security vulnerabilities, while containing as few false positive alerts as possible. To make the solution holistic, all programming languages used by the pilot use case should be RESCALE – Public – Page 25 / 31