Control of Optical Networks: a Reality Check and Future Perspectives

Control of Optical Networks: a Reality Check and Future Perspectives Piero Castoldi(1), Filippo Cugini(2), Alessio Giorgetti(3),(4), Francesco Paolucci(2), Anna Lina Ruscelli(1) Nicola Sambo(1) Andrea Sgambelluri(1), Luca Valcarenghi(1) (1) Scuola Superiore Sant’Anna, Pisa, Italy, piero[email protected] (2) CNIT, Pisa, Italy. (3) University of Pisa, Italy. (4) CNR-IEIIT, Pisa, Italy. Abstract This paper investigates the main highlights of optical network control evolution, focusing on Software-Defined Networking (SDN), NETCONF/YANG protocols, telemetry techniques, advancements in packet/optical networking, and the integration of Artificial Intelligence (AI) within optical networks. ©2024 The Author(s) Introduction Modern optical network control takes advantage of advanced architectural solutions that rely on the progress in network softwarization (SDN, NETCONF/YANG modeling and AI-driven control), monitoring tools (such as telemetry) and tight integration of packet and optical transport. In fact, SDN coupled with protocols like NETCONF and YANG, facilitates standardized configuration and management of network devices, fostering automation and interoperability. Additionally, telemetry empowers real-time monitoring and data collection from network devices, offering granular insights into network performance and health[1]. The convergence of packet and optical networking helps in addressing the ever-increasing demands for higher bandwidth and scalability [2] . This integration leverages coherent technology to bridge the gap between packet-switched and optical transport networks, enhancing efficiency and reducing latency. Furthermore, the advent of pluggable optics and Smart Network Interface Cards (SmartNics) introduces flexibility and intelligence at the edge of networks, optimizing data processing and throughput[3]. Eventually, Machine Learning (ML) and Artificial Intelligence (AI) enable intelligent automation and predictive analytics. AI allows to optimize usage of optical network resources, to predict failures, and dynamically adjust network parameters based on traffic patterns and performance metrics. AI-driven insights enable proactive network management, enhancing reliability and scalability while minimizing operational overhead[4]. In this paper, we summarize the potentials of the aforementioned techniques, examining their implications, challenges, and future prospects in shaping the future optical network infrastructures. SDN empowered by modeling and telemetry In the SDN paradigm, a centralized controller is responsible for all the decisions to be taken in the network, maintaining the topology and configuring all the data-plane devices. In most of the cases, the SDN controller is responsible of the RSA, but recently this task can be delegated to a dedicated component (i.e., digital-twin [5] ) able to assess the QoT of a new lightpath before its configuration. The communication among controller and optical devices can be performed using different protocols (e.g, NETCONF, RESTCONF, gRPC/gNMI) over a dedicated out-of band channel. More specifically, the scientific community converged on the utilization of the NETCONF protocol, that has the important benefit of not requiring extension at the protocol level for transporting new fields. However, relevant work has been conducted on YANG models (e.g., OpenConfig, OpenROADM, T-API and Telecom Infra Project initiatives). This way, an SDN controller can build on standard YANG models and procedures to consistently control, configure and monitor the optical network. In addition, the SDN control of optical networks have evolved in the recent years to support disaggregation, novel network equipment, pluggable optics and programmable packet optical nodes, real time telemetry of Quality of Transmission (QoT) optical parameters. Fig. 1 shows the current state-of-the-art of the SDN control of optical networks encompassing telemetry and AI-assisted re-optimization, following the framework of the zero-touch networking. With the advent of SDN, different open-source initiatives have tried to design and implement a viable controller software. Today, at least three popular transport controller frameworks are available. A first one is based on the open-source OpenDayLigth controller, that is fully compliant with OpenROADM modeling and it has good stability and well tested routines. A second option is Open Network Operating System (ONOS), an open-source framework designed for reliability and scalability, supporting both OpenConfig and Open- Hierarchical controller Node OS Packetoptical node OLS controller Packetoptical node Node OS Kafka bus /gRPC collectors Real time ONT streams AI engines Kafka topics (distributed AI) anomaly detection messages Real time INT streams IP Controller NETCONF/YANG •Control •SetTelemetry TX1 70km 70km A1 A2 A3 A4 70km TX2 70km RX1 RX2 PinA1 PinA2 PinA3 PinA4 L1 L2 WSS WSS B Fig. 1: SDN control and telemetry for AI-driven autonomous optical networking. ROADM models. Recently, a third option appeared, namely TeraFlowSDN that has been extended to enable the control of optical devices, with a preliminary implementation supporting OpenConfig model. In order to control a multi-layer network, composed by optical and packet domains, a hierarchical architecture is typically adopted, with child controllers (respectively an IP controller and an optical controller) coordinated by a parent, that is responsible of the end-to-end workflow. Resorting to the NETCONF/YANG southbound frameworks, thanks to open models such as the OpenConfig/gNMI and OpenROADM initiatives, SDN controllers, Open Line System (OLS) controllers and disaggregated node controllers have the capability to configure the activation of telemetry of inline optical parameters originated by optical cards and pluggables, ROADMs, OXCs and line amplifiers. Typically, QoT telemetry of optical connections is performed out-of-band using control or dedicated channels (i.e., out-of-band network telemetry, ONT). Conversely, in the case of wavelenghts activated between pluggable optics hosted in programmable packet-optical switches, in-band network telemetry (INT) is feasible [6] (see Fig. 1), covering use cases where the reaction time to anomaly telemetry patterns is crucial to guarantee fast local in-band control (e.g., soft failure detected and recovered at the packet-optical node). Effective hard and soft failure detection and localization, and in general anomaly detection of optical performance KPI at the card/amplifier level has become feasible thanks to telemetry correlation. The control of network telemetry at the desired granularity facilitates the release of accurate optical network datasets that are essential to train next-generation AI solutions at the controllers/orchestrators, and in the next years in the network nodes as well [7] , and to feed continuous data analytics for the deployment and the refinement of optical network digital twins. Moreover, to enable third-party data analytics (i.e., for tenants, sliced networking or multi-stakeholder scenarios), optical telemetry streams encoding/decoding techniques have been proposed to ensure accurate machine learning classification while preserving the operator confidential data[8]. In our view, the evolution of control will go beyond the rigidity of NETCONF/YANG and will envision a common programmable model environment enabling massive and decentralized streaming of joint control and telemetry messages continuously assisted by the AI. This updated layer is conceived to cooperate with a novel management and application framework, exploiting explainable and generative AI tools and API (e.g., ChatGTP-like bots) for simplified human-to-system interaction by the operators, capable of mapping semantic commands into a set of automatic control plane and telemetry streams. This will drastically improve the planning, management and maintenance operation, with significant benefits to OPEX. Control of Packet/optical networks The emergence of coherent pluggable transceivers, nowadays available at 800G and beyond also in the QSFP-DD form factor, has boosted the development of IP over Wavelength Division Multiplexing (IPoWDM) solutions. Within this context, the Optical Internetworking Forum (OIF) standardized the Common Management Interface Specification (CMIS), which extends to digital coherent modules (DCO) through C-CMIS (Coherent CMIS). This interface defines the initialization and control of optical modules, serving as the de facto link between routers and coherent pluggable transceivers. The introduction of IPoWDM brings both benefits and new challenges at the network management level. On one hand, it eliminates the necessity for traditional transponders/muxponder devices, leading to reduced capital expenditure (CAPEX), lower latency by bypassing intermediaries, decreased power consumption, and simplified planning, installation, and maintenance processes due to fewer involved elements. Conversely, it necessitates the development of novel SDN architectures to enable coordinated control of IP and optical resources. Historically, IP and WDM configurations have been managed separately, each by its own controllers and teams, with one handling packet resources and the other overseeing optical transport. Within the Telecom Infra Project (TIP), the MANTRA (metaverse ready architectures for open transport) architecture strives to establish a comprehensive hierarchical control architecture for multi-layer networks, taking into account the utilization of IPoWDM nodes. In [9],[10] , two distinct SDN architectures were introduced: Dual and Single. Both solutions rely on the utilization of a parent controller, an IP controller and an optical controller. In both of them only the IP controller is allowed to configure IPoWDM nodes, although in the Dual solution, the optical controller is granted read access to the current configuration of optical pluggables. Demonstration of the MANTRA architecture have been outlined in [11] , showcasing the setup of a point-to-point connectivity service and characterizing the 400 ZR/ZR+ coherent pluggable transceivers in terms of tunability. The obtained results underscore the critical nature of traffic recovery in multi-layer networks using IPoWDM nodes, where all proposed solutions, while successfully validated, exhibit non-negligible complexity. The aforementioned advances in pluggable modules is also opening up new research directions, where edge computing nodes, relying on powerful smartNIC [12]–[14] , can be directly equipped with coherent transceivers. This solution has the potential to drastically reduce the presence of standalone aggregation routers (upper Fig.2) while providing edge nodes with embedded HW-accelerated capabilities for cyber-security and decentralized monitoring (lower Fig.2). Fig. 2: Evolution of optical networking supported by SmartNic ML in Optical networks ML is being investigated to empower the control and management of optical networks. Several use cases for the adoption of ML have been identified [15] , mainly including QoT estimation/prediction, device control (amplifiers), (soft-)failure management, traffic prediction, resource allocation. Generally, ML is a powerful tool when the modelling of a specific phenomena is particularly complex, while ML intrinsically learns relations among training data. ML applied to QoT estimation and prediction has been widely investigated in the literature [16],[17] , e.g. in the context of digital twin (DT) formulations [17] . In parallel, transmission performance models have reached good performance in several data plane scenarios (e.g., Cband transmission): the Gaussian Noise model [18] and its extensions (e.g., the Generalized Gaussian Noise Model) have been widely accepted – e.g. by industrial players within the TIP – and also tested [19] by the optical community. Thus, given the availability of enough-quality transmission models, ML can complement these models to reduce margins and to estimate the input parameters to feed analytical expressions [20] , since physical parameter values typically differ between the reality and the datasheet. Regarding the control of data plane devices, ML can be adopted to characterize devices such as amplifiers [21],[22] , especially when their behaviour is complex to be modelled: an example is Thulium Doped Fiber Amplifier (TDFA) [22] , which is based on three pumps. (Soft-)failure management is another field of application for ML: e.g., it can be exploited for failure prediction [23] , identification [24] , and alarm suppression. Regarding identification, which is typically a classification problem, some failures occur more frequently than others. Consequently, data collected from real networks to train ML models are imbalanced among classes limiting ML performance, as shown in [25] , where data were collected for seven months on a operating network. However, data augmentation techniques [24] can mitigate data class imbalance improving ML performance. Then, a critical event can raise flood of alarms in Network Management System and alarm clustering simplifies the management of alarms by grouping related ones together. ML can assist alarm suppression. Finally, reinforcement learning (RL) and deep reinforcement learning (DRL) have been applied for resource allocation (routing and spectrum assignment) [26] . It has been shown that RL and DRL can achieve good performance, although valid heuristics could be adopted. In general, ML applied to the control and management of optical networks is still under investigation. However, there are relevant limitations for a full awareness of the realistic use cases: data from real networks to feed research activities are scarce because of confidentiality reasons; lab setups can hardly reflect the behavior of a real network given the environment under control (e.g., improbable realistic fiber stresses), a limited number of channels, the setup dimensions, and the reduced observation time windows they are available. Probably, the most promising applications for ML are: i) QoT estimation joining transmission modelling and ML; ii) control of amplifiers; iii) (soft-)failure and alarm management. Finally, optical fibers can act as sensors (e.g., for seismic applications [27] ) and even in this field ML is under investigation[28]. Conclusions Novel capabilities in the control of optical network such as SDN, data plane modeling, advanced control interfaces, telemetry, ML and AI technologies are all essential for building future packet-optical networks. Acknowledgements This work has been partially supported by the EU SNS SEASON Project (101096120) and by the EU under the Italian National Recovery and Resilience Plan (NRRP) of NextGenerationEU, partnership on "Telecommunications of the Future" (PE00000001 - program "RESTART"). Work is carried out within the Department of Excellence in AI and Robotics 2023-2027. References [1] H. Song, G. Luo, and H. J. Chao, “Network telemetry: State-of-the-art and research challenges”, IEEE Communications Surveys & Tutorials, vol. 21, no. 2, pp. 1829– 1853, 2019. [2] M. Glick and J. E. Bowers, “Packet and optical convergence: Technologies, architectures, and trends”, IEEE Network, vol. 35, no. 2, pp. 252–259, 2021. [3] A. Ghiasi, N. McKeown, B. Plattner, A. Ghobadi, A. Wang, and S. Katti, “Smartnics: Accelerating network functions in the cloud era”, IEEE Micro, vol. 40, no. 2, pp. 66–74, 2020. [4] Q. Zhang, Y. Liu, Y. Sun, C. Chen, L. Zhang, and X. Wang, “Ai-driven optimization for dynamic optical network resource management”, Journal of Lightwave Technology, vol. 41, no. 3, pp. 512–527, 2023. [5] G. Borraccini et al., “Experimental demonstration of partially disaggregated optical network control using the physical layer digital twin”, IEEE Transactions on Network and Service Management, vol. 20, no. 3, pp. 2343– 2355, 2023. DOI:10.1109/TNSM.2023.3288823. [6] F. Cugini, C. Natalino, D. Scano, F. Paolucci, and P. Monti, “P4-based telemetry processing for fast soft failure recovery in packet-optical networks”, in 2023 Optical Fiber Communications Conference and Exhibition (OFC), 2023, pp. 1–3. DOI:10.1364/OFC.2023.M1G.2. [7] F. Paolucci, A. Sgambelluri, M. Felipe Silva, et al., “Peerto-peer disaggregated telemetry for autonomic machinelearning-driven transceiver operation”, Journal of Optical Communications and Networking, vol. 14, no. 8, pp. 606– 620, 2022. DOI:10.1364/JOCN.456666. [8] M. F. Silva, A. Sgambelluri, A. Pacini, et al., “Confidentiality-preserving machine learning algorithms for soft-failure detection in optical communication networks”, Journal of Optical Communications and Networking, vol. 15, no. 8, pp. C212–C222, 2023. DOI: 10. 1364/JOCN.481690. [9] O. Gonzales De Dios et al., “Mantra whitepaper ipowdm convergent sdn architecture - motivation, technical definition and challenges”, in Telecom Infra Project White Paper version 3, 2022. [10] A. Sgambelluri, D. Scano, R. Morro, et al., “Failure recovery in the mantra architecture with an ipowdm sonic node and 400zr/zr+pluggables”, J. Opt. Commun. Netw., vol. 16, no. 5, B26–B34, May 2024. [11] R. Morro, E. Riccardi, D. Scano, et al., “First demonstration of mantra ipowdm convergent sdn architecture using sonic white box and 400zr/zr+ pluggables”, in ONDM 2023. [12] P. Castoldi, R. A. Bakar, A. Sgambelluri, J. J. V. Olmos, F. Paolucci, and F. Cugini, “Programmable packet-optical networks using data processing units (dpus) with embedded gpu”, in (OFC) 2024, 2024. DOI: 10.1364/OFC. 2024.Tu3B.4. [13] L. Barsellotti, F. Alhamed, J. J. V. Olmos, F. Paolucci, P. Castoldi, and F. Cugini, “Introducing data processing units (DPU) at the edge”, in ICCCN 2022. [14] F. Cugini, M. Agus, M. Quagliotti, et al., “Point-to-multipoint coherent optics on data processing units (DPUs) for beyond-5G low-latency applications”, in ICTON, 2023. [15] F. Musumeci, C. Rottondi, A. Nag, et al., “An overview on application of machine learning techniques in optical networks”, IEEE Communications Surveys Tutorials, vol. 21, no. 2, pp. 1383–1408, 2019. DOI: 10 . 1109 / COMST.2018.2880039. [16] M. Ibrahimi, H. Abdollahi, C. Rottondi, et al., “Machine learning regression for QoT estimation of unestablished lightpaths”, Journal of Optical Communications and Networking, vol. 13, no. 4, B92–B101, 2021. DOI: 10.1364/ JOCN.410694. [17] D. Sequeira, M. Ruiz, N. Costa, A. Napoli, J. Pedro, and L. Velasco, “OCATA: A deep-learning-based digital twin for the optical time domain”, Journal of Optical Communications and Networking, vol. 15, no. 2, pp. 87–97, 2023. DOI:10.1364/JOCN.477341. [18] P. Poggiolini, G. Bosco, A. Carena, V. Curri, Y. Jiang, and F. Forghieri, “The GN-model of fiber non-linear propagation and its applications”, Journal of Lightwave Technology, vol. 32, no. 4, pp. 694–721, 2014. DOI: 10.1109/ JLT.2013.2295208. [19] A. Nespola, S. Straullu, A. Carena, et al., “GN-Model validation over seven fiber types in uncompensated PM16QAM Nyquist-WDM links”, IEEE Photonics Technology Letters, vol. 26, no. 2, pp. 206–209, 2014. DOI: 10.1109/LPT.2013.2292330. [20] N. Morette, I. F. de Jauregui Ruiz, H. Hafermann, and Y. Pointurier, “On the robustness of a ML-based method for QoT tool parameter refinement in partially loaded networks”, in (OFC) 2022, pp. 1–3. [21] M. P. Yankov, F. Da Ros, U. C. de Moura, A. Carena, and D. Zibar, “Flexible Raman amplifier optimization based on machine learning-aided physical stimulated Raman scattering model”, Journal of Lightwave Technology, vol. 41, no. 2, pp. 508–514, 2023. DOI: 10.1109/ JLT.2022.3218137. [22] M. Radovic, A. Sgambelluri, N. Sambo, and P. Castoldi, “Neural network-based control of TDFA”, in ONDM, 2024. [23] M. F. Silva, A. Pacini, A. Sgambelluri, and L. Valcarenghi, “Learning longand short-term temporal patterns for ml-driven fault management in optical communication networks”, IEEE Transactions on Network and Service Management, vol. 19, no. 3, pp. 2195–2206, 2022. DOI: 10.1109/TNSM.2022.3146869. [24] L. Z. Khan, J. Pedro, N. Costa, L. De Marinis, A. Napoli, and N. Sambo, “Data augmentation to improve performance of neural networks for failure management in optical networks”, Journal of Optical Communications and Networking, vol. 15, no. 1, pp. 57–67, 2023. DOI: 10.1364/JOCN.472605. [25] C. Tremblay, A. Mahmoudialami, P. A. Ngani Sigue, et al., “Detection and root cause analysis of performance degradation in optical networks using machine learning”, in ECOC 2023, 2023, pp. 1278–1281. [26] A. Terki, J. Pedro, A. Eira, A. Napoli, and N. Sambo, “Deep reinforcement learning for resource allocation in multi-band optical networks”, in ONDM, 2024. [27] M. Cantono, J. C. Castellanos, V. Kamalov, et al., “Seismic sensing in submarine fiber cables”, in ECOC 2021, 2021. DOI:10.1109/ECOC52684.2021.9605838. [28] P. D. Hernández, J. A. Ramírez, and M. A. Soto, “Deeplearning-based earthquake detection for fiber-optic distributed acoustic sensing”, Journal of Lightwave Technology, vol. 40, no. 8, pp. 2639–2650, 2022. DOI: 10. 1109/JLT.2021.3138724.