Initial System Architecture

Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Health and Digital Executive Agency (HADEA). Neither the European Union nor the granting authority can be held responsible for them Deliverable due: M9 Actual submission: M9 Lead Beneficiary: CF Dissemination Level PU Public ✓ PP Restricted to other programme participants (including the Commission Services) RE Restricted to a group specified by the Consortium (including the Commission Services) CO Confidential, only for members of the Consortium (including the Commission Services) Open and Programmable Accelerators for Data-intensive Applications in the Cloud Project no: 101189551 Project acronym: CHORYS Horizon Europe RIA HORIZON-CL4-2024-DIGITAL-EMERGING-01-21 Project start date: 1.1.2025 Project duration: 48 months Deliverable D2.4 Initial System Architecture Initial System Architecture Deliverable Information Document Administrative Information Project Acronym: CHORYS Project Number: 101189551 Deliverable Number: D2.4 Deliverable Full Title: Initial System Architecture Deliverable Short Title: Initial System Architecture Document Identifier: CHORYS-D2.4-InitialSystemArchitecture-submitted Lead Beneficiary: CF Report Version: v1.0 Due Date: M9 Report Submission Date: M9 Dissemination Level: PU Nature: Report Lead Author(s): Alberto Lerner Co-author(s): Keywords: cloud systems architecture, accelerators Status: draft, final, x submitted Change Log Date Version Author/Editor Summary of Changes Made 16/09/2025 v0.1 Alberto Lerner (CF) Initial template 05/10/2025 v0.2 Alberto Lerner (CF) First draft 06/10/2025 v1.0 Alberto Lerner (CF) Incorporated suggestions and completed the document Deliverable D2.4 2 of 21 Initial System Architecture Table of Contents Executive Summary ............................................................................................. 6 1. Introduction .................................................................................................. 7 1.1 Motivation........................................................................................................... 7 1.2 Purpose and Scope of the Document....................................................................... 7 1.3 Non-Objectives.................................................................................................... 7 1.4 Structure of the Document ..................................................................................... 8 2. Accelerated Cloud Systems Architecture............................................................. 9 2.1 Traditional Cloud Systems Stacks............................................................................ 9 2.2 The Accelerator Layer........................................................................................... 9 2.3 Integrating the Accelerator Layer............................................................................. 10 3. Accelerated Cloud Management Architecture....................................................... 12 3.1 Traditional Cloud Management Stack ....................................................................... 12 3.2 Managing I/O Accelerators..................................................................................... 13 3.3 Integrating the Accelerator Management................................................................... 13 4. Detailed Flows............................................................................................... 15 4.1 Server-level Flows................................................................................................ 15 4.2 Device-level Flows ............................................................................................... 16 5. Conclusions.................................................................................................. 18 Bibliography ....................................................................................................... 19 Deliverable D2.4 3 of 21 Initial System Architecture List of Figures Figure 1: A traditional layered cloud application (left) and examples of systems at each layer (right). 9 Figure 2: The accelerator access layer can be integrated into the HW/OS layer from Fig. 1. .......... 9 Figure 3: Internal view of the SSD-OS and Menschen. .......................................................... 10 Figure 4: Example of how accelerator-layer features in one layer support features on the next layer. 11 Figure 5: A traditional cloud management stack (right) and exmaples of systems at each layer (left). 12 Figure 6: Extending the fundamental services from Fig. 5 to work on accelerators....................... 13 Figure 7: Example of how accelerator-management features in one layer support features in the next................................................................................................................ 14 Figure 8: In traditional data center servers, the CPU becomes the bottleneck. ............................ 15 Figure 9: Processing flows on accelerators avoid the issue..................................................... 15 Figure 10: Example of how to support the SQL-over-Parquet service in Fig. 4.............................. 16 Deliverable D2.4 4 of 21 Initial System Architecture List of Abbreviations CPU Central Processing Unit GPU Graphics Processing Unit NIC Network Interface Card NDP Near-Data Processing OS Operating System SSD Solid-State Drive Deliverable D2.4 5 of 21 Initial System Architecture Executive Summary Data-intensive applications primarily consist of data flows that access data from storage or network devices and apply a series of transformations to them. The CHORYS project aims at accelerating these applications by a deceptively simple proposition: to allow data flows to perform transformations directly within the storage and network devices. We achieve this by equipping these devices with a combination of slightly modified RISC-V CPUs and specialized hardware units known as accelerators. These accelerators are designed to excel at specific, commonly used data manipulation primitives, referred to as data patterns. In Deliverable D4.1 (“Design Space for RISC-V Extensions”), we outlined our approach to designing and prototyping these data pattern accelerators. In this deliverable, we discuss how the resulting accelerated storage and network infrastructure can serve as a foundation for building new or enhancing existing cloud environments. In summary, we propose integrating our devices into standard rack-based servers commonly used by hyperscalers [5]. We will equip our devices with an access layer similar to that of CPUs, conferring our devices with the ability to load programs, isolate between them (e.g., for virtualization), and monitor their resource consumption (e.g., for billing purposed). This access layer allows hyperscalers to then integrate Near-Data Processing (NDP) capabilities into their service layers. For instance, storage services, such as file systems, could be enhanced to include programs that execute during file read or write operations. Similarly, layers for network communication could utilize this capability when transmitting or receiving data. The anticipated net effect is that by simply swapping out regular devices for our computational ones and adjusting the service layers, the platform will gain computational capacity and improve efficiency. Deliverable D2.4 6 of 21 Initial System Architecture 1 Introduction 1.1 Motivation We are currently facing an unprecedented computational crunch [13]. This situation is primarily driven by the demand for hardware in machine learning, particularly Graphics Processing Unit (GPU), but it also impacts the hardware that supplies data to these GPUs. The CHORYS project is developing two key devices to address this issue: a Solid-State Drive (SSD) known as OpenSSD-V and a Network Interface Card (NIC) called OpenNIC-V. As essential I/O devices, they are designed to store and transport data to GPUs and Central Processing Unit (CPU) efficiently. This is not, however, the primary reason our devices are competitive. What sets them apart is their ability to process data directly, rather than merely transmitting it to CPUs and GPUs for processing, which alleviates the burden on those units. Consequently, using our devices instead of traditional SSDs and NICs enhances a platform’s computational power while maintaining the same physical footprint—among other advantages. How should these new platforms be designed? 1.2 Purpose and Scope of the Document This document lays the groundwork for this question to be answered. Our premise is that the Cloud—large data centers with hundreds of thousands of somewhat uniform rack-mounted servers—is and will continue to be the prevalent architecture to deliver large computational platforms. Our focus, therefore, is on how to position our devices as new building blocks within cloud environments and, importantly, how existing cloud software stacks can integrate and benefit from the features our devices offer. Central to this discussion are the application acceleration capabilities of the CHORYS’s devices. This feature allows our devices to process some very common data manipulation patterns faster than CPUs or GPUs, while consuming less energy. We credit this efficiency to a combination of specialized hardware we are designing, modified RISC-V CPUs that our devices carry, and software primitives to expose them to applications. In this document, we will explore how cloud applications can interact with our primitives and how hyperscalers—the companies behind large cloud infrastructures—can effectively manage a substantial number of our devices. 1.3 Non-Objectives While there are many commonalities across different cloud stacks, each hyperscaler’s offer is peculiar. We are going to refer to random examples of cloud infrastructure software to make the discussion more concrete, but we are not endorsing any vendor or comparing different vendors ’ stacks. We note also that the integration between cloud stacks and our hardware and software infrastructure occurs at a very low level within the stack. The benefits of this integration, however, can propagate to different systems. We focus on integration aspects in this document and mention in high-level terms how it can benefit other systems. It is not our goal to provide detailed designs of the latter. Deliverable D2.4 7 of 21 Initial System Architecture 1.4 Structure of the Document The remainder of this deliverable is organized as follows: Section 2 presents a traditional Cloud Systems stack and discusses the integration of the CHORYS devices within it. Section 3 provides a similar analysis, but this time, it focuses on the Management stack commonly found in cloud environments. In Section 4, we describe the architectural changes that occur within a datacenter-class server, which is the fundamental building block for larger cloud infrastructures. We conclude this document in Section 5, where we summarize our findings and mention further discussions we will present in upcoming deliverables. Deliverable D2.4 8 of 21 Initial System Architecture 2 Accelerated Cloud Systems Architecture 2.1 Traditional Cloud Systems Stacks The services offered by hyperscalers represent just the visible tip of a deep and extensive stack of systems [5]. In Fig. 1, we attempt to simplify this stack for the purposes of our discussion. We do so by organizing these systems into the different classes of applications they enable. The figure presents two examples of applications: a database system and a message queue. We selected these applications because they rely on storage and networking functionalities, respectively, which can benefit from the OpenSSD-V and OpenNIC-V, as we will discuss. Figure 1: A traditional layered cloud application (left) and examples of systems at each layer (right). The applications at the top of the stack leverage functionalities provided by external services that are both visible and purchasable by cloud users. For instance, a database may utilize an object store or a block store, while a message queue likely depends on group communication services. This represents the extent to which external users interact with the cloud. However, the external systems are far from the bottom of the stack. They rely on proprietary systems such as distributed file systems [8], lock servers [6], log servers [21], and remote procedure call functionality [11], among others. These systems remain unseen by external users, and companies often treat them as trade secrets, despite publishing high-level descriptions. These systems stand to gain the most from our devices, providing new hyperscalers a pathway to catch up with established companies. They depend on a commoditized base layer, predominantly featuring an Operating System (OS) (typically Linux) or some form of hypervisor (such as Xen [26]). We propose that by integrating our devices at the base of the stack and enabling them to export services to the upper layers, we can facilitate an advanced stack that runs in parallel with and benefits from a traditional stack. 2.2 The Accelerator Layer Adding the OpenSSD-V and the OpenNIC-V to the base of the stack involves providing resource management features for these devices, similar to an OS, as illustrated in Fig. 2. The OpenNIC-V will enhance a foundational software called Menshen [24]. We will delve into this functionality when discussing the management aspects of the stack. Figure 2: The accelerator access layer can be integrated into the HW/OS layer from Fig. 1. Deliverable D2.4 9 of 21 Initial System Architecture We cannot emphasize enough the magnitude of the potential here, so we quote verbatim from a study about the cost of moving data on scientific applications: “The energy cost of data movement has been identified as one of the major limiting factors for the development of efficient and sustainable exascale systems.” [14]. There is another opportunity for optimizing the data movement. For reductive computations, such as data filtering or aggregation, the output can be significantly smaller than the input. By shifting computations to the device, we transfer the output rather than the input, resulting in a corresponding reduction in data movement, whether the output is consumed by the CPU or another unit on the server. 4.2 Device-level Flows Shifting computations to a device is done by logically attaching a program to an otherwise naturally occurring flow on the device. For instance, using the OpenSSD-V, one might intercept a data flow generated by a read request and apply a transformation to it. Figure 10 illustrates this example and contrasts it with a standard data flow. The standard read flow appears in yellow, showcasing the transfer of data from Flash Memory—a type of persistent memory commonly used in SSDs [16]—to the host. Notably, during this transfer, the data remains unchanged. Figure 10: Example of how to support the SQL-over-Parquet service in Fig. 4 Figure 10 also illustrates how a program can intercept the data flow, depicted in blue. The program running on an OpenSSD-V can operate on this flow in two ways: by utilizing a modified RISC-V CPU that can access the data directly or by leveraging the data pattern accelerators implemented in hardware. This example illustrates both methods in the context of parsing and filtering the data flow. Such capabilities would be particularly helpful for the SQL-over-Parquet application discussed in Section 2.3. The program starts by performing data parsing using the corresponding data pattern accelerator. This parsing process reconstructs data records present in the stream, which are encoded in the Parquet format. The parsed records are temporarily stored in memory, accessible by the RISC-V CPU on the device. For this example, the CPU filters the records by discarding those that do not meet specified criteria. Following this, the program reconstructs the data stream in the Parquet format using another data pattern accelerator, the deparser, ensuring that the resulting stream appears as though the filtered-out records never existed. One may wonder why different mechanisms for program execution are combined on the OpenSSD-V when a general-purpose CPU could suffice. The challenge with relying solely on a CPU is that, to compete with the CPUs found in data center servers, the device’s processor would Deliverable D2.4 16 of 21 Initial System Architecture need to be significantly fast. Our goal is to ensure that programs moved into the device run at comparable speeds. However, incorporating such a high-performance CPU in the SSD would inevitably increase its energy consumption. The advantage of specialized accelerators lies in their potential to be orders of magnitude more energy efficient than a CPU [7]. However, even when our selection of accelerators targets common and costly data manipulation primitives, there remains a possibility that programs may require functionalities outside this scope. By combining the accelerators with a carefully balanced RISC-V, specifically designed for our workloads, we get the best of both worlds. Deliverable D2.4 17 of 21 Initial System Architecture 5 Conclusions In this document, we presented how to integrate the CHORYS accelerated devices into a typical cloud architecture from principles. We discussed separately how to integrate our technology into applications and the management stacks typically found in larger hyperscalers. The integration was discussed at several levels of detail: from how to use our devices in an individual server, to how to run programs in these devices, to how this ability could create new and enhance existing cloud services. The main takeaways from these discussions are as follows: • A cloud provider does not need to alter its architecture to adopt our innovations completely. • They can modify existing or add a new platformand internal-level services that utilize the NDP features of the OpenSSD-V and the OpenNIC-V. • For applications, this means faster performance and the ability to perform a larger volume of data. • For cloud providers, our technology can provide a competitive edge vis-a-vis more established or larger hyperscalers. This document also laid the groundwork for discussions we will have in upcoming deliverables. It introduced the abstraction of data flows and showed examples of how to improve them using our technology. Still missing are a more complete definition of data flows, where they appear, and how to attach logic to them, which will demonstrate further why we believe they are central to our innovations. We also expect to discuss the topics we presented here in light of more specific cloud architectures (e.g., Cyso’s architecture, which is one of CHORYS Project’s partners, or IPCEI-CS reference architectures [12]). Deliverable D2.4 18 of 21 Initial System Architecture References [1] Apache Parquet. https://parquet.apache.org. [2] ARMBRUST, M., GHODSI, A., XIN, R., ZAHARIA, M., ET AL. Lakehouse: a new generation of open platforms that unify data warehousing and advanced analytics. In Proceedings of CIDR (2021), vol. 8, p. 28. [3] AWS CloudWatch. https://aws.amazon.com/cloudwatch/. [4] AWS S3 Select: Querying data in place with amazon s3 select. https://docs.aws.amazon. com/AmazonS3/latest/userguide/selecting-content-from-objects.html. [5] BARROSO, L. A., HÖLZLE, U., AND RANGANATHAN, P. The datacenter as a computer: Designing warehouse-scale machines. Springer Nature, 2019. [6] BURROWS, M. The Chubby lock service for loosely-coupled distributed systems. In Proceedings of the 7th symposium on Operating systems design and implementation (2006), pp. 335–350. [7] DALLY, W. J., TURAKHIA, Y., AND HAN, S. Domain-specific hardware accelerators. Communications of the ACM 63, 7 (2020), 48–57. [8] GHEMAWAT, S., GOBIOFF, H., AND LEUNG, S.-T. The google file system. In Proceedings of the nineteenth ACM symposium on Operating systems principles (2003), pp. 29–43. [9] Google Cloud Dashboard. https://cloud.google.com/monitoring/dashboards. [10] Google gFlags. https://gflags.github.io/gflags/. [11] gRPC: A high performance, open source universal rpc framework. https://grpc.io. [12] IPCEI next generation cloud infrastructure and services (IPCEI CIS). https://commission.europa.eu/projects/ ipcei-next-generation-cloud-infrastructure-and-services-ipcei-cis-secunet-secunetedgecloud-essen_ en. [13] JIN, B. OpenAI unveils plans for seemingly limitless expansion of computing power. The Wall Street Journal, 2025. [14] KESTOR, G., GIOIOSA, R., KERBYSON, D. J., AND HOISIE, A. Quantifying the energy cost of data movement in scientific applications. In 2013 IEEE international symposium on workload characterization (IISWC) (2013), IEEE, pp. 56–65. [15] Kubernetes Cluster Manager. https://kubernetes.io. [16] LERNER, A., AND BONNET, P. Principles of Database and Solid-State Drive Co-Design. Springer, 2025. [17] MQ Telemetry Transport format and protocol. https://www.ibm.com/docs/en/ibm-mq/9.3. x?topic=reference-mq-telemetry-transport-format-protocol. [18] PAL, K. K., AND SUDEEP, K. Preprocessing for image classification by convolutional neural networks. In 2016 IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT) (2016), IEEE, pp. 1778–1781. [19] PORTS, D. R., AND NELSON, J. When should the network be the computer? In Proceedings of the Workshop on Hot Topics in Operating Systems (2019), pp. 209–215. Deliverable D2.4 19 of 21 Initial System Architecture [20] ROSEN, R. Resource management: Linux kernel namespaces and cgroups. http://www. haifux.org/lectures/299/netLec7.pdf, 2013. [21] SIGELMAN, B. H., BARROSO, L. A., BURROWS, M., STEPHENSON, P., PLAKAL, M., BEAVER, D., JASPAN, S., AND SHANBHAG, C. Dapper, a large-scale distributed systems tracing infrastructure. Tech. rep., Google Inc., 2010. [22] TIRMAZI, M., BARKER, A., DENG, N., HAQUE, M. E., QIN, Z. G., HAND, S., HARCHOLBALTER, M., AND WILKES, J. Borg: the next generation. In Proceedings of the fifteenth European conference on computer systems (2020), pp. 1–14. [23] VERMA, A., PEDROSA, L., KORUPOLU, M., OPPENHEIMER, D., TUNE, E., AND WILKES, J. Large-scale cluster management at google with borg. In Proceedings of the tenth european conference on computer systems (2015), pp. 1–17. [24] WANG, T., YANG, X., ANTICHI, G., SIVARAMAN, A., AND PANDA, A. Isolation mechanisms for High-Speed Packet-Processing pipelines. In 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22) (2022), pp. 1289–1305. [25] WILKINSON, J. Site reliability engineering: how Google runs production systems. "O’Reilly Media, Inc.", 2016, ch. Practical Alerting from Time-Series Data. [26] Xen: The world’s most secure, stable and performant open-source hypervisor. https: //xenproject.org. Deliverable D2.4 20 of 21 Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Health and Digital Executive Agency (HADEA). Neither the European Union nor the granting authority can be held responsible for them Consortium Disclaimer All information provided reflects the status of the CHORYS project at the time of writing and may be subject to change. Neither the CHORYS Consortium as a whole, nor any single party within the CHORYS Consortium warrant that the information contained in this document is capable of use, nor that the use of such information is free from risk. Neither the CHORYS Consortium as a whole, nor any single party within the CHORYS Consortium accepts any liability for loss or damage suffered by any person using the information. Copyright Notice © 2025 by the authors, the CHORYS Consortium. This work is licensed under a “CC BY 4.0” license.