CXL Memory Management for the CMS L1 Scouting System and Beyond

CXL Memory Management for the CMS L1 Scouting System and Beyond AUGUST 2024 AUTHOR: Guilherme Paulino CMS L1 Data Scouting SUPERVISORS: Giovanna Lazzari Miotto Thomas Owen James CERN openlab Report // 2024 i CXL Memory Management for the CMS L1 Scouting System and Beyond ABSTRACT The Level 1 (L1) Trigger at CMS performs fast reconstruction in FPGA logic, processing a subset of detector data at the 40 MHz bunch-crossing rate ("scouting”). This process selects the most interesting events for full read-out and processing in the software filter farm at a frequency of approximately 100 kHz. The emerging Compute Express Link (CXL) protocol operates over the PCIe physical layer and dynamically multiplexes IO, cache, and memory protocols. It is engineered to substantially reduce latency in heterogeneous and disaggregated computing through efficient resource sharing, shared memory pools, and enhanced data and operand movement between accelerators and target devices. In the context of ongoing collaboration efforts with Micron Technology, we aim to leverage CXL for the L1 Scouting ingestion and data processing chain, providing coherent and seamless access to buffered data from multiple processors and compute accelerators. Ultimately, the CXL-powered system will offer low-latency access to both raw and processed CMS data in a short-term staging area that precedes tape storage. This project explores the feasibility and compatibility of integrating CXL into CMS's existing L1 Scouting system by working with demonstrator and test systems installed at CERN, assessing the protocol's potential to enhance the L1 Scouting process with efficient, cache-coherent data handling at scale, thereby contributing to the system's potential for CMS research. With the High Luminosity LHC upgrade and the expected increase in data volumes, the CMS Level-1 Scouting System requires advanced event selection and data processing techniques. And for this reason, this research investigates innovative solutions to address these needs, focusing on latency, bandwidth, and memory management challenges through the evaluation of CXL-enabled devices. By combining CXL’s cachecoherency capabilities with the Fabric-attached memory file system (Famfs), the system can achieve disaggregated shared files and multi-host memory management. This ongoing integration within the SCDAQ software aims to demonstrate the effectiveness of these technologies in enhancing the performance and scalability of CMS's data processing framework. CERN openlab Report // 2024 ii CXL Memory Management for the CMS L1 Scouting System and Beyond TABLE OF CONTENTS 1. INTRODUCTION 1 L1 DATA SCOUTING OVERVIEW CXL MEMORY INTERFACES OBJECTIVES 2. MEMORY LAKE CONCEPT 3 FAMFS FILE SYSTEM 3. EXPERIMENTAL SETUP 4 SERVER ENVIRONMENT MICRON MEMORY EXPANSION 4. BENCHMARK RESULTS AND ANALYSIS 5 PERFORMANCE MEASUREMENTS MEMORY LATENCY CHECKER STREAM MULTICHASE STRESSAPPTEST 5. DISCUSSIONS 9 REFERENCES 10 CERN openlab Report // 2024 1 CXL Memory Management for the CMS L1 Scouting System and Beyond 1. INTRODUCTION The Compact Muon Solenoid (CMS) experiment at CERN's Large Hadron Collider (LHC) is tasked with analyzing the massive amount of data generated by particle collisions occurring at a rate of 40 million events per second (40 MHz). A challenge lies in the fact that it is not feasible to fully read out and process data from all detectors at this rate due to limitations in bandwidth and storage, which would require more than 480 Tb/s, but also for the physical characteristics of the detectors such as readout time or power consumption. To manage this, the Level 1 (L1) Trigger system is employed to quickly identify and select the most interesting events for further analysis, reducing the data rate from 40 MHz to approximately 100 kHz. The CMS Level 1 Data Scouting (L1DS) is an advanced technique used within this framework to acquire and analyze data that would typically be discarded by the L1 Trigger. It involves the nonstandard use of the trigger for data acquisition, allowing researchers to access a broader range of physics by bypassing some of the constraints imposed by L1 latency and accept rate. This approach enables the CMS experiment to explore additional signatures and conduct more comprehensive analyses, thereby enriching its overall physics program [1]. As the LHC undergoes upgrades, particularly with the forthcoming High-Luminosity upgrade, the volume of data expected to be generated will significantly increase. This motivation presents new challenges in data acquisition, processing, and storage. Addressing these challenges will require the integration of emerging technologies, such as AI acceleration and real-time data indexing, into the existing data scouting infrastructure. These technologies will be crucial in maintaining and enhancing the CMS experiment’s ability to process and analyze the ever-growing data volumes efficiently. L1 DATA SCOUTING OVERVIEW The L1DS system is designed to capture and process data at the full 40 MHz bunch crossing rate, focusing on a subset of the most valuable trigger primitives. The data acquisition process begins with FPGA boards that receive, align, and pre-process the data streams before transmitting them via high-speed Ethernet connections to a cluster of computing nodes. These nodes, running an Intel TBB-based data acquisition software (SCDAQ), perform further data reduction and online processing using the CMS reconstruction framework [2]. However, as data volumes continue to grow, the system faces significant challenges, particularly in managing high data throughput and maintaining low-latency processing. Efficient data handling and processing are essential to ensuring that the L1DS system can scale effectively to meet the demands of the High-Luminosity LHC, allowing CMS to continue its pursuit of physics analyses that might otherwise be constrained by traditional trigger limitations. Figure 1 presents the current L1DS infrastructure main components and protocols. CERN openlab Report // 2024 2 CXL Memory Management for the CMS L1 Scouting System and Beyond Figure 1. Diagram of the CMS L1 Data scouting computing infrastructure currently in operation. Source: [3]. CXL MEMORY INTERFACES The Compute Express Link (CXL) [4] protocol represents a transformative technology in the realm of heterogeneous and disaggregated computing. As a new alternate protocol capable of running over the standard PCIe physical layer, CXL dynamically multiplexes IO, cache, and memory protocols, enabling highly efficient resource sharing, shared memory pools, and enhanced data movement between accelerators and target devices. This protocol is specifically designed to significantly reduce latency, which is critical in high-performance computing environments such as the CMS L1 Scouting System. By integrating CXL into its online processing infrastructure, CMS aims to leverage these advanced capabilities to enhance the L1 scouting data processing pipeline. The ability to access data coherently and seamlessly from multiple processors and compute accelerators through CXL's shared memory architecture promises to improve processing efficiency and scalability, addressing the increasing data volumes expected from future LHC upgrades. OBJECTIVES The primary objective of the project is to integrate Micron CXL-enabled memory devices into the CMS L1DS system's ingestion and data processing chain. This integration aims to enhance the system by providing coherent and seamless access to buffered data across multiple processors and compute accelerators, significantly improving data handling efficiency. Additionally, the project seeks to achieve low-latency access to both raw and processed data within a short-term storage environment, thereby ensuring scalability and responsiveness in managing the increasing data volumes expected from the High-Luminosity LHC upgrades. IU (DAQ) Ingestion Units x 8 PUPUIU Processing Units data streams over 100 Gbps TCP/IP AU AU Accelerator Units SU SU SU Storage Units Tier 1 Local Storage < 2 GB/s CERN openlab Report // 2024 3 CXL Memory Management for the CMS L1 Scouting System and Beyond 2. MEMORY LAKE CONCEPT This proposal was designed to address the growing challenges in handling massive data volumes generated by systems like the CMS L1 Scouting System. As data volumes continue to increase, traditional memory architectures face limitations in terms of scalability, latency, and bandwidth. The Memory Lake architecture, show in Figure 2, proposes a solution that leverages disaggregated memory resources, creating a shared memory pool that can be accessed by multiple hosts and accelerators. This architecture supports efficient, high-bandwidth, low-latency access to memory, enabling better data management and processing in environments that demand fast, large-scale data transfers. By utilizing shared memory across multiple systems, this architecture facilitates more efficient memory utilization, minimizes data duplication, and improves overall processing efficiency. Figure 2. Diagram of the Memory Lake concept. Source: [3]. At the top, a pool of byte-addressable memory is accessible via CXL, enabling seamless access to shared memory across the entire system. Below the memory pool, CXL switches facilitate dynamic connectivity between various system components, including Ingestion Units (IUs), Processing Units (PUs), Accelerator Units (AUs), and Storage Units (SUs). The ingestion units receive high-speed data streams over 400 Gb/s optical links, which are processed by the processing and accelerator units before being stored in tiered storage, providing efficient data handling and improved processing capabilities. Combining the use of disaggregated shared files, using a fabric-attached file system, with cache-coherent CXL for byteaddressable memory would allow the creation of a memory lake with capacities in the order of hundreds of terabytes. FAMFS FILE SYSTEM The Fabric-Attached Memory File System (FAMFS) [5] is a key component in the Memory Lake architecture, providing the software infrastructure needed to manage disaggregated memory. FAMFS is a file system that allows shared memory to be accessed by multiple hosts, using memory mapping and DAX devices without the need for intermediate processor caching. This direct access capability ensures low-latency memory transactions, making it particularly well-suited for applications that require rapid memory access or the use of hardware accelerators, like GPUs or FPGAs. Byteaddress able memory Byteaddress able memory Byteaddress able memory Byteaddress able memory Byteaddress able memory IU (DAQ) Ingestion Units x 16 PUPUIU Processing Units data streams over 400 Gbps TCP/IP AU AU Accelerator Units SU SU SU Storage Units Tier 1 Local Storage Tier 2 Local Storage < 100 GB/s < 5 GB/s Permanent Storage CERN openlab Report // 2024 4 CXL Memory Management for the CMS L1 Scouting System and Beyond 3. EXPERIMENTAL SETUP SERVER ENVIRONMENT The experimental setup for the LD1S demonstrator of the integration with CXL technology is built on a hardware and software environment designed for high-performance data ingestion and processing. The server hardware comprises a Supermicro system equipped with two AMD EPYC 9454 CPUs, each offering 48 cores and capable of a peak bandwidth of 460 GB/s per socket, making it well-suited for handling largescale data tasks. The system also includes two Micron CZ120 CXL Type 3 memory expansion modules, each with 256 GB DDR4 memory, supporting CXL.io and CXL.mem protocols, and delivering up to 36 GB/s of memory bandwidth with low latency. The operating system is based on a FAMFS-enabled Linux kernel (RHEL 9.3 with kernel 6.8-rc4), which facilitates efficient memory management and seamless integration of CXL memory devices within the data processing chain. Figure 3 presents a photo of the server internals. Figure 3. CXL host node at CMS USC. Server supplied by E4 Computer Engineering. Two CZ120 memory expansion modules provided by Micron. Source adapted from: [1]. i. MICRON MEMORY EXPANSION The Micron CZ120 memory expansion module is a high-performance CXL Type 3 device designed to enhance memory capacity and bandwidth in advanced computing environments. Available in 128GB and 256GB configurations, the CZ120 leverages Micron's DDR4 technology, supporting data transfer speeds of up to 3200 MT/s. It operates over a PCIe Gen5 x8 interface, delivering a maximum bandwidth of up to 36 GB/s with a read latency as low as 90 nanoseconds under unloaded conditions. The module also features advanced RAS (Reliability, Availability, Serviceability) capabilities, including single error correction and double error detection (SECDED) and single device data correction and error correction code (SDDC ECC) for error correction, thermal throttling, and secure boot features. The CZ120 is designed for seamless integration into high-density computing systems, providing a robust solution for applications requiring large, high-speed memory resources. CZ120 CERN openlab Report // 2024 5 CXL Memory Management for the CMS L1 Scouting System and Beyond 4. BENCHMARK RESULTS AND ANALYSIS This section presents the benchmark results obtained from the experimental setup and carries out analysis of memory performance metrics such as latency, bandwidth, and throughput, as well as the comparison of results with expectations and previous benchmarks. PERFORMANCE MEASUREMENTS i. MEMORY LATENCY CHECKER The Memory Latency Checker (MLC) [6] is a tool developed by Intel to measure and analyze memory latency and bandwidth in computing systems. It is particularly useful for evaluating the performance of memory subsystems, including those involving advanced technologies like CXL. MLC can perform both unloaded and loaded latency tests, providing insights into how memory latency changes under different bandwidth conditions. For example, it allows us to observe how higher bandwidth demands can lead to increased latency, which is crucial for optimizing system performance. The tool supports various configurations, enabling detailed assessments of memory behavior in environments such as utilizing the Micron CZ120 memory expansion modules. In this context, MLC helps determine the efficiency of memory operations, ensuring that the system meets the stringent performance requirements for high-speed data processing. The measurements obtained results are shown in Figure 4. Figure 4. Latency measured at different bandwidth points. These results were obtained using the Memory Latency Checker tool for one CZ120 memory module expansion device. As Figure 4 indicates, latency remains relatively low and stable across most bandwidth levels but begins to increase sharply as the system approaches its maximum bandwidth capacity, particularly around 25 to 35 GB/s. The steep rise in latency at higher bandwidths is most pronounced in the 0:1 and 1:1 read to nontemporal write ratios, which is typical as higher bandwidth demands lead to increased memory contention and reduced efficiency. CERN openlab Report // 2024 6 CXL Memory Management for the CMS L1 Scouting System and Beyond Conversely, mixed read-write workloads (represented by the pink and blue points) show more balanced performance, with lower latency at higher bandwidths compared to non-temporal write-dominant workloads. This demonstrates the importance of optimizing memory access patterns to maintain lower latency and achieve higher overall performance in systems utilizing the CZ120 memory module. ii. STREAM The STREAM [7] benchmark is a widely used synthetic benchmark designed to measure sustainable memory bandwidth (in MB/s) and the corresponding computation rate for simple vector operations. The benchmark consists of four primary functions, where each function tests different aspects of memory performance: - Copy: Measures the rate of copying data from one location to another (C = A). - Scale: Tests the rate of scaling a vector by a constant and storing the result (B = scalar x C). - Add: Measures the rate of adding two vectors and storing the result (C = A + B). - Triad: Combines scaling and addition in a single operation (A = B + scalar x C). Figure 5 presents the results of the STREAM benchmark across four different configurations. Figure 5. STREAM results for each memory device configuration. In direct access (DAX) mode, the memory can be accessed directly without using the processor cache. The performance varies across functions, with the highest bandwidth observed in the Triad operation (22.67 GB/s) and the lowest in the Copy operation (15.34 GB/s). The average times are relatively consistent, though the Add operation has a slightly higher average time. NUMA (Non-Uniform Memory Access) mode represents the performance when memory is accessed through a traditional DRAM configuration, where memory access time varies depending on the processor's proximity to the memory node. The bandwidth in NUMA mode is higher across all functions compared to DAX mode, with Triad and Add operations reaching approximately 23.97 GB/s. This mode also shows the lowest average times, reflecting an expected slightly better efficient memory access. DAX Mode using SIMD (Single Instruction, Multiple Data) optimization is applied to improve parallel processing, allowing multiple data points to be processed with a single instruction. This mode shows an improvement over standard DAX mode, especially in the Copy operation, which reaches a bandwidth of 19.91 GB/s, a significant increase compared to the unoptimized DAX mode. The average times are slightly lower than standard DAX mode, indicating that SIMD optimization effectively improves performance, particularly in operations that can benefit from parallel processing.