Challenges in Deterministic Networking Implementation for Cloud-Edge Continuum Computing Platform

Challenges in Deterministic Networking Implementation for Cloud-Edge Continuum Computing Platform Kohei Taniguchi†♯, Arata Endo∗, Hirotake Abe‡, Chonho Lee§, Takahiro Hirofuchi¶, Ryousei Takano¶, Takaaki Fukai¶, Akram Bem Ahmed¶, Takeharu Kato¶, Susumu Date† †Osaka University,∗Nara Institute of Science and Technology,‡University of Tsukuba,§Okayama University of Science, ¶National Institute of Advanced Industrial Science and Technology (AIST) ♯[email protected] I. BACKGROUND A Post 5G (P5G) network realizes services that take advantage of its low latency and high concurrent connectivity to provide real-time feedback to the real world. Such services require a P5G network to provide an infrastructure that guarantees the services can complete their service delivery, which is feedback to their users using data collected from geographicallydistributed IoT devices, within a specified period. To meet the requirement, a P5G network has to possess enough network resources to ensure that the services run punctually without disruption. However, since the deployment cost of such an infrastructure would be high, its utilization cost would also be high. To reduce the cost, it is important to develop a Platform as a Service (PaaS) that enables a P5G network to accommodate many services by efficiently allocating network resources to them. As such a PaaS, we plan to develop Cloud-Edge Continuum Computing Platform (CECC-Platform), which supports the time multiplexing of network resources. The idea behind the time multiplexing is that it is wasteful to always allocate network resources to a service that periodically runs a service delivery triggered by a data collection. Based on this idea, CECC-Platform allocates network resources to a service in a time-slot manner so that the network resources are available to other services while the service delivery of the service does not run. In other words, from the perspective of a service, network resources are allocated to itself right before its service delivery starts and they are released as soon as the service delivery is completed. In this way, CECC-Platform can provide the infrastructures for more services at a lower cost even though it has limited network resources. To realize the time multiplexing in CECC-Platform, it is required to precisely estimate the completion time of the service delivery including data transfer on a network in advance. This estimation is difficult since the service delivery on a best-effort network is not deterministic. On the other hand, recently, the specification development of Deterministic Networking (DetNet), which is a protocol suite that constructs an L3 network with guaranteed end-to-end latency, is in progress. In DetNet, the completion time can be easily estimated considering the guaranteed latency. Therefore, we will adopt DetNet for CECC-Platform. In this paper, we aim to reveal what challenges we have to tackle in the implementation of DetNet on CECC-Platform and to discuss our research direction. This paper is structured as follows. Section II describes the mechanism to guarantee latency in DetNet. In Section III, we derive challenges from the requirements of DetNet in CECC-Platform. Based on the challenges, we discuss our research direction in Section IV. Finally, Section V concludes this paper. II. OVERVIEW OF DETERMINISTIC NETWORKING DetNet guarantees end-to-end latency by controlling the packet transmission timing at each switch to prevent packet collisions. The DetNet implementation consists of regulators and queue scheduling in highly accurate time-synchronized switches. The combination of regulators, queue scheduling, and time synchronization method determines the upper bound of the end-to-end latency of DetNet. Time synchronization between switches is achieved by copying the reference clock to each switch via communications. Since the CPU frequency variation shifts the switch clocks, it is required to copy the reference clock frequently to maintain highly accurate time synchronization. An alternative method is frequency synchronization which synchronizes the switch CPU frequency to the reference clock CPU frequency. The regulators, such as Token-Bucket Filter and Interleaved Regulator, limit the ingress traffic at switches. By limiting the ingress traffic, they prevent the switch buffer from filling up and eliminate packet loss. The regulators prevent packet loss in such a way and guarantee the end-to-end latency is finite. The queue scheduling controls the transmission timing of packets from the queue. For example, Time-Aware Shaper (TAS) prevents starvation and guarantees end-to-end latency with Gate Control List (GCL) that opens and closes queue exits according to time [1]. Asynchronous Traffic Shaper (ATS) uses Diffserv-based priority control to guarantee latency for communications in high-priority classes [2]. TAS and ATS guarantee that packets are sent out within a specific time, resulting in a guaranteed end-to-end latency. In general, TAS has a smaller upper bound of latency than ATAS. However, TAS requires high-accurate time synchronization and can guarantee latency only for fewer flows. III. CHALLENGES IN THE DETERMINISTIC NETWORKING IMPLEMENTATION As mentioned in Section II, the upper bound of end-to-end latency in DetNet depends on the combination of regulators, queue scheduling, and time-synchronization method. To host real-time services by CECC-Platform, from the perspective of a service delivery deadline, we consider the combination of a TAS and high-accuracy time synchronization is better for DetNet on CECC-Platform. To realize CECC-Platform by using DetNet with this combination, there are the following two challenges. 1) Handle the degradation of time synchronization accuracy. 2) Increase the number of flows guaranteed latency. Challenge 1 is derived from the communication frequency for high-accuracy time synchronization in DetNet. Since we assume the number of switches in DetNet become large, it is not feasible to synchronize all switch clocks accurately in terms of traffic amount and switch load. The regulators and queue scheduling methods depend on the local switch clock. If the local switch clock is not synchronized, the regulator and queue scheduling cause malfunctions [3]. As a result, DetNet fails to guarantee end-to-end latency. To maximize the number of services hosted by CECCPlatform, Challenge 2 must be addressed. The number of flows for which DetNet guarantees latency directly relates to the number of services hosted by CECC-Platform. However, as mentioned in Section II, TAS can guarantee latency for a few flows. The main reason is the shortage of bandwidth and time slots in the switches. As the number of flows guaranteed latency increases, no bandwidth or time slots are available in the switch to allocate to the new flows. Therefore, optimizing switch resource allocation is required to increase the number of flows guaranteed latency in DetNet. However, optimizing the resource allocation across all switches is recognized as an NP-hard problem and is challenging to solve. IV. RESEARCH DIRECTION We plan to tackle the following three research topics to solve these challenges mentioned in Section III. 1) Lightweight and high-accuracy time synchronization for switches. 2) Mitigation for the effects of time synchronization error. 3) Autonomous distributed resource allocation optimization among switches. Topic 1 and 2 solve Challenge 1. For highly accurate time synchronization among many switches, it is necessary to reduce the communication frequency for time synchronization. To reduce the communication frequency, we focus on frequency synchronization as mentioned in Section II. Under frequency synchronization, once the switch clocks are synchronized, it keeps the switches as time-synchronized for a while. Therefore, in Topic 1, we try to reduce the frequency of time synchronization by selecting frequency-synchronized switches and creating time-synchronized pairs. However, even if time synchronization as in Topic 1 is performed, time synchronization errors are unavoidable due to thermal throttling and time stamp errors. Therefore, DetNet requires mechanisms to mitigate the effects of such time synchronization errors. For example, when the time synchronization errors cause packet loss or increase latency, it is possible to recover the guaranteed latency through readjustment of regulators and queue scheduling according to the switch status. Topic 3 solves Challenge 2. Although the resource allocation optimization for all switches is an NP-hard problem as mentioned in Challenge 2, the optimization for a switch is easy. Therefore, it is divided into two optimization problems: intra-switch resource allocation optimization and inter-switch allocation coordination. In this way, we believe the computation time for the optimizations can be kept small even if the number of switches and flows increases. V. CONCLUSION This paper introduced the challenges in the DetNet implementation for CECC-Platform. For CECC-Platform to host more services using time-multiplexing, a highly timedeterministic network like DetNet is useful. We derived the challenges when using DetNet on CECC-Platform: time synchronization of switches and scalability in the number of flows. Finally, we discussed research directions for these challenges. For the challenge of time synchronization, we presented the utilization of frequency synchronization and the mitigation mechanism. We suggested an autonomous distributed resource allocation optimization for the challenge of scalability in the number of flows. We plan to tackle these challenges. ACKNOWLEDGMENT This paper is based on results obtained from “Research and Development Project of the Enhanced Infrastructures for Post5G Information and Communication Systems” (JPNP20017), commissioned by the New Energy and Industrial Technology Development Organization (NEDO). REFERENCES [1] Zhao Luxi, Pop Paul, and Craciunas Silviu S, “WorstCase Latency Analysis for IEEE 802.1 Qbv Time Sensitive Networks Using Network Calculus,” IEEE Access, vol. 6, pp. 41 803–41 815, 2018. [2] Zhou Zifan, Berger Michael St¨ ubert, Ruepp Sarah Ren´ ee, and Yan Ying, “Insight into the IEEE 802.1 Qcr Asynchronous Traffic Shaping in Time Sensitive Network,” Advances in Science, Technology and Engineering Systems Journal, vol. 4, no. 1, pp. 292–301, 2019. [3] Thomas Ludovic and Le Boudec Jean-Yves, “On Time Synchronization Issues in Time-Sensitive Networks with Regulators and Nonideal Clocks,” Proceedings of the ACM on Measurement and Analysis of Computing Systems, vol. 4, no. 2, pp. 1–41, 2020.