Breaking the I/O Barrier: 1.2 Tb/s Ethernet Packet Processing on a GPU

Breaking the I/O Barrier: 1.2 Tb/s Ethernet Packet Processing on a GPU John W. Romein Euro-Par'25 August 27th–29th, 2025 2 Outline •radio astronomy & radio telescopes •GPU signal processing & I/O challenge •Grace Hopper, Data Plane Development Kit + GPUdev •results 3 Challenges in astronomy •understanding the universal laws of physics –fast radio bursts –dark matter, dark energy –when did the first stars start to shine? –how do supermassive black holes and galaxy clusters shape the universe? –habitability around low-mass stars? –directly detect exoplanets? –... build telescopes→ 4 Why observing in radio? Gamma rays, X-rays and ultraviolet light blocked by the upper atmosphere (best observed from space). Visible light observable from Earth, with some atmospheric distortion. Most of the infrared spectrum absorbed by atmospheric gases (best observed from space). Radio waves observable from Earth. Long-wavelength radio waves blocked. 0 % 50 % 100 % 0.1 nm 1 nm 10 nm 100 nm 1 µm 10 µm 100 µm 1 mm 1 cm 10 cm 1 m 10 m 100 m 1 km Wavelength Atmospheric opacity credits: NASA •atmosphere transparent •some phenomena only visible at radio wavelengths •H I line 5 Problem •long wavelengths •high-resolution images large telescopes→ Solution •combine data from many receivers •use earth rotation 6 Processing pipeline local processing combine data (real time) calibration, imaging, etc. disk local processing local processing “correlator” WAN 7 Processing pipeline local processing combine data (real time) calibration, imaging, etc. disk local processing local processing “correlator” WAN FPGAs or GPUs FPGAs CPUs, GPUs 8 Processing pipeline local processing combine data (real time) calibration, imaging, etc. disk local processing local processing “correlator” WAN this talk's focus GPUs 9 Data flow ●digitizer network (corner turn) correlator→ → ●typically: UDP ●stream data directly FPGA GPU→ 16 Three approaches GPUCPUNIC NIC NIC DPDK pktmbufs flat ring buffers DMA DMA GPUCPUNIC DMA NIC NIC flat ring buffers DPDK pktmbufs GPUCPUNIC NIC NIC pktmbuf payloads ring buffer pktmbuf headers DMA DMA ring buffer 2-copy 1-copy 0-copy ●split packet –header CPU, payload GPU→ → ●ring buffer = DPDK packet buffer 17 Performance measurements 0 200 400 600 800 1000 1200 network input bandwidth (Gb/s) peak network bandwidth 2-copy 1-copy 0-copy 0 200 400 600 800 1000 1200 network input bandwidth (Gb/s) 0 100 200 300 CPU memory bandwidth use (GB/s) peak memory bandwidth 2-copy 0-copy •only 0-copy achieves 1.2 Tb/s •bypassing CPU memory is essential 18 GPU frequency tuning •436 W 342 W→ 1.98 1.86 1.74 1.62 1.50 GPU clock speed (GHz) 0 50 100 execution time (ms) real-time run-time limit correlate not real time 1.98 1.86 1.74 1.62 1.50 GPU clock speed (GHz) 0 100 200 300 400 GPU power (Watt) 19 CPU frequency tuning •119 W 69 W→ 3.6 3.3 3.0 2.7 2.4 2.1 1.8 1.5 CPU clock speed (GHz) 0.0 0.2 0.4 0.6 0.8 1.0 fraction of time spent DPDK rte_eth_rx_burst() DPDK rte_pktmbuf_free() application code not real time (packet loss) 3.6 3.3 3.0 2.7 2.4 2.1 1.8 1.5 CPU clock speed (GHz) 0 50 100 CPU power (Watt) 6 cores 20 Are we happy with the results? •DPDK GPUdev approach: ☺ exceptionally high data rates  CPU + GPU overhead  0-copy difficult to integrate into existing correlator software •future work: explore alternatives 21 Summary •GPUs correlate 100x faster in ~10 years –compute challenge I/O challenge→ •combine hardware & software innovations –Grace Hopper/NVLink –3x 400 GbE –DPDK GPUdev •GPU handles Ethernet packets –1.2 Tb/s 22 Acknowledgements •This work was funded by –European Southern Observatory (ALMA GPU Correlator Study) –EU HORIZON INFRA-TECH (RADIOBLOCKS, Grant Agreement nr. 101093934) –Netherlands eScience Center (PADRE, RECRUIT) –NWO Netherlands Foundation for Scientific Research (DAS-6)