Data Processing Unit (DPU) and P4 Programmability in Support of the Edge Continuum

Data Processing Unit (DPU) and P4 Programmability in Support of the Edge Continuum Filippo Cugini(1), Rana Abu Bakar(2), Andrea Sgambelluri(2), Nicola Sambo(2), Lorenzo De Marinis(2), Alessio Giorgetti(1)(3)(4), Piero Castoldi(2), Juan Jose Vegas Olmos(5), Francesco Paolucci(1) (1) CNIT, Pisa, Italy. filippo[email protected] (2) Scuola Superiore Sant’Anna, Pisa, Italy, (3) University of Pisa, Italy. (4) CNR-IEIIT, Pisa, Italy. (5) NVIDIA, Denmark. Abstract A comprehensive framework for pervasive telemetry and accelerated networking is presented. The framework is suitable for effective integration of optical, packet, and computing resources, leveraging DPU and P4 programmability in support of edge-to-edge continuum. ©2024 The Author(s) Introduction Recent advances in transmission technologies and network programmability are driving the effective integration of optical, packet, and computing resources in support of edge-to-edge and edgeto-cloud continuum [1],[2] . At the transmission level, coherent pluggable transceivers in small form factors are boosting the development of IPoWDM switches effectively integrating optical and packet resources in a single network element, with remarkable benefits in terms of capital expenditures, latency, and power consumption [3]–[7] . In addition, coherent pluggables have the potential to be inserted directly into latest generation high-speed Smart Network Interface Card (SmartNIC), also called Data Processing Unit (DPU), further reducing opto-electro-optical conversions [8]–[10] . Both switches and DPU can also leverage network programmability and hardware acceleration of networking functions (e.g., using P4 or DOCA technologies, respectively). In this paper, we present and discuss a comprehensive framework for pervasive telemetry and accelerated networking, suitable for effective integration of optical, packet, and computing resources, leveraging DPU and P4 programmability in support of edge-to-edge continuum. Integrated computing and IPoWDM programmable network scenario Fig. 1 shows the futuristic yet achievable reference scenario including edge computing resources equipped with DPU (E1-E3) and IPoWDM switches equipped with programmable ASIC (N1N3). Both DPU and IPoWDM switch exploit coherent pluggable modules. This scenario assumes the overcoming of few technological limitations, such as the hardware compatibility of DPU to host coherent transceivers. The integrated scenario enables the deployment of a scalable pervasive telemetry system, exploiting the offloading of telemetry packet processing to the programmable IPoWDM switches and DPUs. Furthermore, it provides the capability to rapidly react against critical events (e.g., failure or SLA degradations) by dynamically enforcing network reconfigurations, without requiring the intervention of SDN Controllers. In-network telemetry for HW-accelerated failure detection Fig. 1 shows a network of three packet-optical nodes interconnected through optical line systems. The connectivity between N1 and N3 is assumed to be served by a pair of unidirectional lightpaths (for simplicity, only one shown in the figure). An alternative route is also available, passing through N2. Each node is aware of the Quality of Transmission (QoT) of the lightpaths terminated at its coherent receivers (e.g., received OSNR, pre-FEC BER, RX power). In addition, each node has access to the packet level Quality of Service (QoS) parameters (e.g., in/out packet/bit rates, queue occupation). For example, N1 monitors QoT and QoS of the optical connections originated at N2 and N3. That is, N1 can immediately detect a (soft) failure affecting N3-N1 link. However, with traditional SDN-based mechanisms, N1 has no visibility on possible QoT degradation affecting the lightpath in the reverse direction (N1-N3) or affecting non-directly connected links (e.g., N2-N3). Indeed, notification methods through a centralized SDN system might not be scalable and sufficiently fast to avoid data losses. In [11] we introduced a pervasive decentralized telemetry system which relies on HW offloading. In particular, it exploits direct, in-network generation and processing of Multi-Layer Network Telemetry (MLNT) information exchanged among packetoptical nodes. MLNT packet reports provide detailed real-time performance (i.e., QoT and QoS) of those links that are of interest for a specific node. For example, N1 can receive telemetry data from N2 and N3, including the N2-N3 connection. Extracting information from the data plane to generate telemetry packet still requires softwarelevel operation. On the other hand, the processing telemetry packets is performed in-network by the EDGE NODE E3 EDGE NODE E1 EDGE NODE E2 WSS WSS WSS WSS WSS WSS DNN DNN DNN LB Edge node state (CPU, mem) Link QoT INT(OSNR, Px, preFEC BER) DOCA N1 N2 N3 123 12 2 1234 DOCA Service INT DOCA DPU1 DPU2 DPU3 Packet-optical network domain N1->N3 N3->DPU3 N2->N3 N3->DPU3 N2->N3 N1->N2 N3->DPU3 N3->DPU3 N2->N3 N1->N2 DPU1->N1 Fig. 1: Programmable Packet optical network connecting Edge computing nodes. P4 ASIC, operating at wire speed within around 1 microsecond. In the proposed framework, the SDN Controller only pre-computes the alternative route(s) to be enforced upon a QoT threshold violation for each active lightpath in the network. Each MLNT packet occupies between 60 and 100 bytes. The overall bandwidth consumed by MLNT packets depends on the generation rate. For example, 0.08 Mb/s is obtained in the case of a MLNT packet every 10ms. Even higher rates (e.g., 1 pck/ms) would still occupy less than 0.001% of a 100 Gb/s connection. In these measurements, we limited the forwarding of MLNT data to 1-hop distances. Edge Continuum via IPoWDM P4-based load balancer Fig. 1 also shows a service request originated at Edge E1, traversing the previously described packet-optical nodes, and executed either at Edge E2 or Edge E3. By exploiting the aforementioned decentralized system, performance degradation due to network issues can be significantly reduced. However, no information is still shared about computing resources. In [12] , we enhanced the previously described telemetry system by encompassing telemetry data originated at the edge computing level. For example, Edges E2 and E3 exchange telemetry packets reporting on their status of CPU and memory usage. These packets are processed by the intermediate Node N3 which is programmed with a P4-based Load Balancer. The Load Balancer includes a parser, two pipelines, and two main P4 registers. In particular, a Flow Table is used with the aim of updating the state of the edge nodes in terms of current CPU load and to dynamically identify the least-loaded destination edge node for the service traffic (UDP traffic is assumed, so MAC and IP destination addresses are updated according to the selected edge node). Results showed that the intra-switch latency is around 250 µ s in software switch implementations, where the impact of P4-based stateful steering is an approximately 50 µ s latency with respect to plain forwarding, corresponding to the additional register read operation performed by both the least-loaded action. However, it is important to highlight that no latency increase can be appreciated using P4 ASIC. To assess the telemetry system with latencysensitive 5G applications, a remote vehicle control application leveraging serverless functions has been deployed. Load balancing at the data plane on a per-packet basis has been successfully demonstrated between two edges without the intervention of the SDN controller. Results showed effective dynamic and beneficial distribution of computational load with negligible increase of application latency with respect to static forwarding towards a single edge. The per-packet load balancing ensures that edge nodes are not overloaded, thus avoiding unpredictable latency increase peaks during serverless function execution due to excessive CPU utilization. This way, the load balancer reduces the jitter and acts as an end-to-end latency stabilizer. In-network DNN for HW-accelerated failure classification and prediction In this section, we further enhance the proposed threshold-based telemetry and control system by introducing a distillation technique that enables in- Tab. 1: Compressing metrics for soft-failure identification Metric bits Resolution OSNR [dB] 8 0.11 dB pre-FEC BER = 6 for x, 0.156 for x, x×10−y3 for y1 for y network Deep Neural Network (DNN) processing at wire-speed. Programmable hardware pipelines lack the arithmetic capabilities required for DNN computations, as they are optimised for match-action with flow tables. In our work [13] we introduced a lossless look-up table distillation technique that maps quantized DNNs (i.e., neural networks based on integers) into flow-tables. The method leverages the quantization feature as an advantage: integerencoded inputs are aggregated and treated as a compound address. The table is generated by simply collecting the DNN outputs for all the possible input configurations. With this method, the inference of a DNN is reduced to a match-action on a flow-table. The method is effective regardless of the DNN depth and complexity, provided that there is enough memory for such a table to be deployed, e.g., for a DNN with two 8-bit inputs and one 8-bit output are required 2 8+8 = 16 kbytes. To validate this method, we trained some quantized DNN for a DDoS mitigation problem, considering the UNSW-NB15 Network Intrusion Detection data set. We reached an F1-score of 93.75% considering 6 stateless features that can be directly extracted from packets. Such a method can be applied also for DNNbased soft-failure identification, relying on the endto-end optical signal-to-noise ratio (OSNR) and pre-forward-error-correction bit error rate (pre-FEC BER) [14] monitored at coherent receivers. A possible compression of OSNR and pre-BER values is reported in Tab.1. Assuming OSNR in [dB] expressed as a decimal number and assuming to saturate its value to 30 dB, if OSNR is expressed with 8 bits, a resolution of 0.11 dB can be achieved. Assuming pre-FEC BER expressed as x×10−y , with x a decimal comprised between 0 and 10 and y an integer, if x is expressed with 6 bits a resolution of 0.156 is achieved, while if y is expressed with 3 bits up to 10−8 can be achieved. Thus, 8 bits are considered for OSNR and 9 for pre-FEC BER, implying 131072 entries in the flow table. It is important to highlight that modern chipsets (e.g., Broadcom Tomahawk) support even 1 million of entries. Future works will investigate the impact of such compression in the accuracy of ML inference, as a function of the number of bits. End-to-end DPU-based monitoring The previous sections report on dynamic telemetry and forwarding operations performed by intermediate transit switches. In this section, we enhance the framework by including the SmartNICs/DPUs at the source and destination nodes (e.g., E1-E3 in Fig. 1). Three DPUs, potentially equipped with coherent pluggable modules, are integrated into the system to handle the data processing tasks, ensuring high throughput and low latency required for real-time processing. We leverage the DOCA Flow API, which enhances the system’s ability to manage network flows dynamically while maintaining the integrity and performance of data processing. This way, all network elements are able to inject telemetry headers (e.g., including OSNR and pre-FEC BER within the UDP payload), enabling detailed monitoring at the packet level. The DOCA Flow API uses Deep Packet Inspection (DPI) accelerations available at the DPU hardware to inspect the packet payload of UDP packets containing telemetry headers from DPU3, DPU2, N3, N2, and N1. DPU1 extracts telemetry information from the payload of identified packets payload using DPI as well. The DPU1 then processes the extracted telemetry data and removes the telemetry headers. Cleaned packets are sent to the end host using the DOCA Flow API for efficient packet forwarding in hardware offload mode. DOCA Telemetry Service (DTS) supports telemetry related to endto-end applications or DPU-related metadata (i.e., service INT as depicted in Fig. 1). DTS monitoring counters (i.e., eth_rx_bytes, eth_tx_bytes, eth_rx_packets, and eth_tx_packets) analyze the network traffic patterns and ensure that the network interfaces are handling the expected load. DTS also provides error detection counters, packet drop counters, and CRC counters for monitoring. In addition, the state of DPU (e.g., the load of its internal ARM-based CPU cores) may be monitored and exchanged in a decentralized environment (e.g., in federated learning scenarios), allowing improved task distribution on the actual accelerated and computing resources. The deep packet inspection performance of a 100G DPU have been tested considering specific telemetry payload matches. Results showed that hardware-accelerated DPI enables regular expression match at rates even higher than 30Gb/s, largely exceeding the requirements for pervasive telemetry systems over converged computing, packet, and optical infrastructures. Conclusions This paper presented a comprehensive framework for pervasive telemetry and accelerated networking. The framework enables effective integration of optical, packet, and computing resources, leveraging DPU and P4 programmability in support of edge-to-edge continuum. Acknowledgements The work on P4 technology has been supported by the EU B5G-OPEN Project (G.A., 101016663). The work on DPU has been supported by the EU SNS SEASON Project (101096120) and by the EU under the Italian National Recovery and Resilience Plan (NRRP) of NextGenerationEU, partnership on "Telecommunications of the Future" (PE00000001 - program "RESTART"). Work is carried out within the Department of Excellence in AI and Robotics 2023-2027. References [1] B. Niu, J. Kong, S. Tang, Y. Li, and Z. Zhu, “Visualize your ip-over-optical network in realtime: A p4-based flexible multilayer in-band network telemetry (ml-int) system”, IEEE Access, vol. 7, pp. 82 413–82 423, 2019. [2] I. Pelle, F. Paolucci, B. Sonkoly, and F. Cugini, “P4assisted seamless migration of serverless applications towards the edge continuum”, Future Generation Computer Systems, vol. 146, pp. 122–138, 2023, ISSN: 0167-739X. DOI: https : / / doi . org / 10 . 1016 / j . future.2023.04.010 . [Online]. Available: https:// www . sciencedirect . com / science / article / pii / S0167739X23001450. [3] S. K. Dey, J. Gupta, and A. Adhya, “Design of capex and opex aware ip-over-wdm network using different types of traffic bypass and grooming techniques”, Optical Fiber Technology (OFC) Conference, 2023. [4] A. Gumaste, A. Napoli, J. Pedro, W. Wakim, and H. Bock, “Cost-effective ip-over-dwdm aggregation and improved router bypass using p2mp optics”, in 49th European Conference on Optical Communications (ECOC 2023), vol. 2023, 2023, pp. 1059–1062. DOI: 10.1049/icp. 2023.2434. [5] A. Sgambelluri, D. Scano, R. Morro, et al., “Failure recovery in the mantra architecture with an ipowdm sonic node and 400zr/zr+pluggables”, J. Opt. Commun. Netw., vol. 16, no. 5, B26–B34, May 2024. [6] E. Pincemin and O. Renais, “Interoperable coherent wdm interfaces at 400g and 800g”, in Optical Fiber Communication (OFC) Conference, 2024. [7] F. Cugini, D. Scano, A. Giorgetti, et al., “Telemetry and ai-based security p4 applications for optical networks”, Journal of Optical Communications and Networking, vol. 15, no. 1, A1–A10, 2023. [8] P. Castoldi, R. A. Bakar, A. Sgambelluri, J. J. V. Olmos, F. Paolucci, and F. Cugini, “Programmable packet-optical networks using data processing units (dpus) with embedded gpu”, in Optical Fiber Communication (OFC) Conference, 2024. [9] Y. Yan, A. F. Beldachi, R. Nejabati, and D. Simeonidou, “P4-enabled smart nic: Enabling sliceable and servicedriven optical data centres”, Journal of Lightwave Technology, vol. 38, no. 9, pp. 2688–2694, 2020. [10] H. Singh, C. Huang, M. Sicard-Gagne, et al., “Intsdn: Evaluation of various p4 parameters using optical telemetry having reconfigurable data plane on 40 gbps line rate”, in 2019 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (PACRIM), IEEE, 2019, pp. 1–6. [11] F. Cugini, C. Natalino, D. Scano, F. Paolucci, and P. Monti, “P4-based telemetry processing for fast soft failure recovery in packet-optical networks”, in 2023 Optical Fiber Communications (OFC) Conference, 2023. [12] I. Pelle, F. Paolucci, B. Sonkoly, and F. Cugini, “P4-based hitless faas load balancer for packet-optical network edge continuum”, in Optical Fiber Communication (OFC) Conference, 2023. [13] E. Paolini, L. D. Marinis, D. Scano, and F. Paolucci, “Inline any-depth deep neural networks using p4 switches”, IEEE Open Journal of the Communications Society, vol. early access, pp. 1–1, 2024. DOI: 10.1109/OJCOMS. 2024.3411071. [14] L. Z. Khan, J. Pedro, N. Costa, L. De Marinis, A. Napoli, and N. Sambo, “Data augmentation to improve performance of neural networks for failure management in optical networks”, Journal of Optical Communications and Networking, vol. 15, no. 1, pp. 57–67, 2023.