Applications and Lessons Learned from AI/ML-Driven SDN Control and Management in Optical Transport Networks

Applications and Lessons Learned from AI/ML-Driven SDN Control and Management in Optical Transport Networks Ricardo Mart´ ınez, Carlos Hern´ andez-Chulde, Ramon Casellas, Ricard Vilalta, Raul Mu˜ noz, Josep M. F` abrega Centre Tecnol` ogic de Telecomunicacions de Catalunya (CTTC/CERCA) Castelldefels (Barcelona), Spain [email protected] Abstract—Effective autonomous SDN control and management functions are becoming increasingly essential for efficiently handling the lifecycle of optical connectivity services, including creation, restoration, updates, and termination, while continuously meeting stringent requirements such as bandwidth, latency tolerance, and reliability. These autonomous functions aim to optimize network performance objectives, such as efficient resource utilization (e.g., optical spectrum and transceivers) and minimizing power consumption. This work introduces a comprehensive architecture, encompassing building blocks, APIs, and workflows, that fully integrates AI/ML models (such as Deep Reinforcement Learning) to support automated SDN control decisions, including dynamic routing and resource selection. We then explore how this architecture addresses key challenges in traditional optical transport networks, such as latency-aware service provisioning, online restoration of connectivity services, energy-efficient resource management, and joint cloud and network resource allocation. Through the exploration of various applications, we present selected results demonstrating how AI/ML-driven solutions can enhance network operations in these areas. Additionally, we share key lessons learned from these applications, providing insights into the benefits and limitations of adopting AI/ML models for autonomous SDN operations in optical transport networks. These lessons focus on practical considerations, challenges in adapting DRL solutions to specific networking problems, and recommendations for optimizing their integration into real-world optical network infrastructures. Index Terms—SDN controller, Deep Reinforcement Learnign, Elastic Optical Networks I. INTRODUCTION In the context of autonomous Software-Defined Networking (SDN) control for transport optical networks, there is a growing interest among network operators in integrating and leveraging advanced Artificial Intelligence (AI) and Machine Learning (ML) strategies. These approaches aim to achieve a wide range of objectives, including enhanced Quality of Transmission (QoT) estimation, failure prediction, improved This work has received funding from the EU’s Horizon Europe research and innovation program, SEASON project, grant agreement no. 101096120. It is partially funded by RELAMPAGO grant PID2021-127916OB-I00 funded by MCIN/AEI/10.13039/501100011033/FEDER/UE and by ERDF A way of making Europe, and ”Ministerio de Asuntos Econ´ omicos y Transformaci´ on Digital” and the European Union-NextGenerationEU in the frameworks of the ”Plan de Recuperaci´ on, Transformaci´ on y Resiliencia” and of the ”Mecanismo de Recuperaci´ on y Resiliencia” under references TSI-063000-2021-22, TSI063000-2021-23, TSI-063000-2021-115. overall network performance, etc. [1]. These autonomic control operations supported by an SDN controller encompass the lifecycle management of optical connectivity services, including provisioning, updating, and restoration. These operations can be achieved and further enhanced by leveraging the assistance of dedicated AI/ML-trained models. In this regard, recent efforts have focused on exploiting Deep Reinforcement Learning (DRL). Notably, a significant number of studies have utilized DRL to train agents that optimize routing and resource selection decisions when provisioning new connectivity services, re-optimizing existing connections, or updating lightpath attributes (e.g., operational modes) to meet diverse service requirements [2], [3]. Specifically, these works explore the application of DRL mechanism from distinct perspectives: efficient spectral resources utilization [3]–[5], improved selection of operational modes (i.e., modulation formats) at the transceivers [6] or QoT-aware routing in Elastic Optical Networks (EONs) [5], [7] or triggering proactive spectrum defragmentation techniques as done in [8]. In this paper, we focus on exploring various applications developed by the authors that leverage DRL strategies, particularly to assist the SDN-driven decision-making processes of Routing and Spectrum Assignment (RSA) algorithms for EON infrastructures. These applications address heterogeneous RSA challenges, such as effectively meeting stringent service latency requirements, ensuring sustainable service provisioning through energy-efficient RSA decisions, enabling online connectivity service restoration, and facilitating joint cloud and network resource allocation for network services. Prior to this, we present the considered and implemented architecture of the AI/ML-assisted SDN controller [9]. Finally, we outline key lessons learned that must be considered when adapting DRL solutions to specific networking problems, with a particular focus on optical transport networks and related technologies. II. ARCHITECTURE FOR AI/ML-INTEGRATED SDN The increasing dynamism of optical networks and the disaggregation of optical infrastructure have driven the development of optical SDN control that integrate AI/ML techniques for network control and management functions towards network automation. A visual representation of the SDN control plane architecture is given in Fig. 1. This illustrates a comprehensive overview of the system’s components, functionalities, and control interfaces between them. The architecture consists of: •SDN Controller: Responsible for the entire service lifecycle, including processing new service requests, finding and allocating the necessary EON resources (e.g., optical spectrum), configuring the involved network elements (e.g., ROADMs), and releasing resources upon service completion. •ML-Assisted Path Computation Element (PCE): A dedicated entity that handles RSA decisions, enabling flexible and scalable execution of RSA algorithms. It hosts the trained DRL agent and interacts with the SDN controller via the TAPI RESTful API. •Northbound Interface (NBI): Provides an API for OSS/BSS or end-users to request connectivity services. The NBI implementation is based on the ONF T-API 2.1 [10], built upon the RESTCONF protocol and YANG models. •Southbound Interface (SBI): Enables the SDN controller to configure the underlying network devices (e.g., ROADMs) to set up the connectivity service. Each device has a SDN agent to handle the allocation or release of the selected resources. Topology Services Latency RSA Energy RSA VNF PlacementRestoration OSS/BSS NBI SBI REST SDN Controller ML-Assisted Functions Fig. 1. SDN Control Plane Externalizing some functions to a dedicated entity offers several advantages, such as enhanced algorithm flexibility, allocation of specific resources for computationally intensive processes, and the ability to easily update and execute algorithms (e.g., RSA, VNF Placement, restoration) in a modular fashion without affecting the SDN controller. The strategic integration of DRL within the SDN architecture underscores a significant advance towards achieving autonomous network operations. DRL agents, trained through interactions within a simulated or real network environment, learn to make routing and resource allocation decisions that optimize network performance metrics such as throughput, energy consumption, and latency. The capability of DRL to dynamically adapt to changing network conditions and service demands, without explicit programming, offers a powerful tool for enhancing network automation. This intelligent decisionmaking process leverages continuous feedback from the network’s current state reported by the SDN controller. The training phase of the DRL models focuses on maximizing cumulative rewards learning optimal policies that aligns with network performance objectives. III. APPLICATIONS OF AI/ML IN SDN-CONTROLLED OPTICAL NETWORKS In this section, we provide an overview of selected applications studied by the authors that adopt AI/ML-trained approaches to assist SDN control decisions, such as routing and resource selection strategies, within optical transport networks. A. Latency-Aware Service Provisioning RSA exemplifies resource allocation and routing challenges, which are considered decision-making tasks in the ML field and can be addressed using DRL approaches. DRL enables the network to adaptively and intelligently manage spectrum resources, optimizing the fulfillment of service requirements (e.g., latency and bandwidth) in real time. The increasing demand for low-latency services, driven by applications such as 5G/6G communications and industrial automation necessitates considering latency as a vital parameter for service provisioning in optical networks. These latency-sensitive applications require guaranteed end-to-end latency bounds to function correctly, making traditional RSA algorithms, which primarily focus on bandwidth allocation, insufficient. In [9], [11], we propose a latency-aware RSA based on DRL. A key aspect of DRL-based RSA is the design of the agent’s observation and action space. In latency-aware service provisioning, the network state representation can include various features, such as the source and destination nodes, the required number of Frequency Slots (FSs), the current spectrum utilization (including position of the first available FS block, average size of available FS blocks, number of available FSs), the maximum tolerated latency and the end-to-end delay of each candidate path. The DRL agent operates with a discrete action space, selecting one of the candidate paths as the feasible path for each request. The reward function is designed to take into account the bandwidth and latency requirements of the connectivity service. The reward encourages the allocation of higher bandwidth and prioritizes lower-latency connections, since it is defined as the sum of a component proportional to bandwidth demand and an inverse component of latency for successful allocation, while a negative reward penalizes failures. Extensive simulation results, comparing the DRL-based approach with traditional heuristics like k-shortest path firstfit (kSPFF) using two network topologies, demonstrate the superiority of DRL in latency-aware service provisioning. Figure 2 shows that the DRL agent consistently achieves lower bandwidth blocking ratios (BBR), especially under high traffic loads. This performance advantage stems from the DRL agent’s ability to learn optimal path selection strategies based on the current network state and predicted traffic patterns. 50 75 100 125 150 175 200 225 250 Erlangs 0.000 0.025 0.050 0.075 0.100 0.125 0.150 0.175 0.200 BBR kSPFF-RSA BCNNet DRL-RSA BCNNet kSPFF-RSA BTNet DRL-RSA BTNet Fig. 2. Performance benchmarking of kSPFF and DRL RSA Furthermore, the DRL-based approach exhibits significant improvement in fulfilling stringent latency requirements compared to kSPFF, as shown in Fig. 3. The DRL agent successfully accommodates a larger number of connections with stricter latency demands, demonstrating its effectiveness in handling latency-sensitive applications. This behavior arises from the DRL agent’s training process, where latency is incorporated as a crucial parameter in the reward function, encouraging the selection of paths that minimize end-to-end delay. 30 35 40 45 50 55 60 Time (ms) 0.010 0.015 0.020 0.025 0.030 0.035 0.040 Probability Density kSPFF DRL Fig. 3. PDF showing the failed services with respect to end-to-end latency demands B. Online Service Restoration Optical networks are the backbone of global communication infrastructure, tasked with delivering vast amounts of data. A key area for development is failure management, as outages in optical connections can significantly impact many users, applications, and services. There is increasing interest in using ML to automate various control operations in optical networks, particularly in failure management. Previous research has explored ML strategies that are both proactive (e.g., performance monitoring and failure prediction) and reactive (e.g., failure detection, localization, and identification) [12]. However, the application of ML, especially DRL, in the restoration process of disrupted lightpaths remains underexplored. This presents a significant opportunity to enhance the reliability and efficiency of optical networks through intelligent automation. In [13], we propose a DRL agent is to select the best restoration sequence from options provided by the Global Concurrent Restoration (GCR) algorithm [14] during a link failure. When a link failure is detected, the DRL agent retrieves the GCR’s candidate solutions and evaluates them using its neural networks. This evaluation considers as observation space factors like the total number of restored lightpaths, total restored bandwidth, and the classification of restored and unrestored lightpaths by their required frequency slots. The DRL agent’s action selects the optimal restoration sequence. The DRL agent’s reward mechanism is based on the amount of bandwidth restored, encouraging solutions that maximize network restorability. The restoration capabilities of the proposed DRL agent is compared with the GCR heuristic algorithm and a sequential approach under various traffic loads and failure durations. The RL agent showed improved restorability over the sequential approach and slightly outperformed the GCR method in low traffic scenarios (200 Erlangs), as ilustrated in Fig. 4. However, in terms of BBR, the DRL agent only surpassed the GCR’s performance without outperforming the sequential method. The improvements in restorability achieved by the DRL agent did not consistently lead to better BBR performance across all methods. This could be because the restorability values were already high and similar among the training samples used for the agent. 0.952 0.953 0.954 0.955 0.956 0.957 0.958 0.959 0 250 500 750 1000 1250 1500 1750 2000 Restorability Episodes Restorability 0.046 0.048 0.05 0.052 0.054 0.056 0.058 0 250 500 750 1000 1250 1500 1750 2000 BBR Episodes BBR DRL GCR Sequential Fig. 4. Restorability and BBR using RL agent, GCR and Sequential Approach. C. Improving Energy Efficiency Network sustainability is currently considered a critical objective to improve the balance between transport capacity (i.e. throughput) and overall power consumption [15]. In transport EONs, a common approach is to develop and implement heuristic energy-aware Routing and Spectrum Assignment (EA-RSA) algorithms [16]. These algorithms aim to provision optical connections that meet the network requirements, such as guaranteed bandwidth and latency, while simultaneously minimizing network power consumption. This creates a multiobjective routing and resource allocation problem, where prioritizing service provisioning to improve BBR performance may compromise energy efficiency (i.e., increase power consumption), and vice versa. In this context, in [15], we proposed an EA-RSA approach that takes advantage of DRL to achieve a superior tradeoff between network throughput and power consumption. The trained DRL EA-RSA agent is thoroughly compared against two baseline heuristics: KSP-FF, which prioritizes connection provisioning without considering power consumption, and a modified EA-KSP-FF, which focuses on minimizing power consumption during the provisioning of new connectivity services. In the proposed DRL EA-RSA agent, the observation space encompasses not only the source and destination nodes of the connectivity request but also other relevant network state parameters. These metrics include parameters related to the computed kshortest paths, such as the end-to-end available spectrum, the existing power consumption of all devices along the kth path, and the amount of requested spectral resources. The discrete action space comprises the set of feasible (up to K) paths between the source and destination that meet the bandwidth requirements. The reward function is designed to simultaneously encourage increased throughput while minimizing network power consumption (e.g., avoiding the activation of ”slept” devices such as ROADMs, optical amplifiers, etc.). As shown in both Fig. 5 and Fig. 6, the DRL EA-RSA results indicate that the trained DRL agent excels in adapting to network conditions, achieving an enhanced tradeoff between network performance and power consumption when compared to both KSP-FF and EA-KSP-FF heuristics.. The work presented in [15] has been recently extended in [17] to encompass not only optical line system devices and elements, such as ROADMs and optical amplifiers, but also coherent DWDM transceivers (pluggables). These transceivers support diverse operational modes by combining various modulation formats and symbol rates, further broadening the scope of the study D. Cloud/Network Resource Allocation Provisioning network services over an inter-DC EON is a significant challenge due to the need to orchestrate both computing resources in data centers (DCs) and spectrum resources on optical links. Network services, modeled as Service Function Chains (SFCs) or VNF Forwarding Graphs (VNFFGs), require careful resource allocation to ensure efficient operation. Efficient resource allocation is crucial to avoid service Fig. 5. Performance on Blocking Bandwidth Ratio (BBR) vs HT (s) Fig. 6. Performance on Av. Network Power Consumption (kW) vs HT (s) blocking, which occurs when either computational resources in the DCs or network resources for interconnecting lightpaths are insufficient. AI/ML techniques, specifically DRL, offer a promising approach for addressing the complexity of VNF placement over SDN-controlled optical networks. We proposed some DRL-based solutions for VNF placement over EONs that integrate techniques such as invalid action masking [18], [19] and Graph Neural Networks (GNNs) [20]. In these works the observation state captures a comprehensive view of the network environment to guide the agent’s decisionmaking process. This includes the available computing resources (i.e., CPU, memory, storage) in each DC, the latency of the best lightpaths between DCs, these lightpaths need to satisfy bandwidth and latency requirements of the virtual links connecting VNFs. The VNF requirements also are included in the observation space as the demand for computing resources and the bandwidth and latency constraints of the virtual links. Integrating GNNs into the DRL agent architecture allows for a more effective representation of the inherent topological features of the network, leading to improved decision-making in VNF placement, since GNNs excel at processing graph- structured data. In the context of VNF placement, the action space involves selecting a DC to host each VNF and choosing a lightpath from a set of candidate paths to interconnect the selected DCs. The reward function employed in these works is defined as follows: a reward of 0 for each successfully allocated VNF, a reward equal to the aggregate computing capacity allocated upon complete network service deployment. This encourages the allocation of complete network services rather than individual VNFs, negative reward if a VNF cannot be deployed due to computing resource constraints in the DCs or unmet networking requirements in the EON. Experimental results demonstrate significant performance improvements over a heuristic baseline algorithm. The DRLbased solution effectively reduces the network service blocking rate, particularly when dealing with higher request loads, and is even better when GNN is incorporated, as shown in Fig. 7. This improvement is attributed to the DRL agent’s ability to learn optimal placement strategies that consider both compute and network resource constraints. Fig. 7. Network Service (NS) Blocking for heuristic, DRL and GNN-DRL Another important aspect of DRL-based VNF placement is its generalization capability. Experimental results, depicted in Fig. 8, suggests that policies learned in one network topology can be effectively applied to different topologies, demonstrating the adaptability of DRL-based solutions. Furthermore, continuous learning techniques can be employed to adapt trained models to changes in topology, allowing the DRL agent to maintain its performance in evolving network environments. IV. LESSONS LEARNED FROM DEPLOYING AI/ML MODELS Exploring AI/ML-based (or assisted) SDN control and management within optical transport networks has provided several key lessons. These insights are crucial for understanding the potential benefits and challenges of integrating AI/ML techniques into network operations. Notably, the careful design and selection of environmental components, such as the State and Action spaces, as well as the Reward function in the learning framework, can significantly influence both the convergence Fig. 8. Network Service (NS) Blocking Rate NSFNet and Unseen topology time of the training process and the overall performance of the trained agent. Furthermore, the choice of additional parameters, including the number of layers and neurons in the Deep Neural Network, learning rate, discount factor, and others, plays a pivotal role in shaping the agent’s effectiveness and efficiency. It is well-known that varying these parameters is a timeand memory-consuming process. Therefore, we specifically focus below on outlining the relevance and impact of carefully selecting the so-called agent environmental parameters. State and Action Space: As mentioned above, the design of the DRL agent’s observation and action spaces plays a crucial role in shaping its learning capabilities and overall performance. A well-crafted state representation that incorporates relevant network features—such as link and path optical spectrum utilization/availability, end-to-end aggregated latency information, and the actual power consumption of network devices along the candidate paths—enables the agent to make more informed and effective decisions. Similarly, the action space should offer enough flexibility to allow the agent to effectively explore and exploit various path selection strategies. Moreover, it is crucial for the DRL algorithm to incorporate invalid action masking, preventing the agent from selecting actions (e.g., candidate paths) that are known in advance to be infeasible. For example, paths with end-to-end latency exceeding the maximum threshold requested by the service should be automatically excluded from consideration. This approach ensures that the agent focuses its learning on viable options, improving both efficiency and performance. Therefore, careful consideration of these design choices is crucial for developing effective DRL-based solutions to assist SDN operations in autonomously managing transport optical networks. These choices directly influence the agent’s ability to learn efficiently and make optimal decisions in complex and dynamic network environments. Reward Function as a Guiding Force: The reward function is pivotal in steering the DRL agent’s learning process by providing feedback on the quality of its actions. A thoughtfully designed reward function encourages the agent to achieve spe- cific network objectives, such as maximizing bandwidth utilization, minimizing latency, or balancing resource efficiency with energy consumption. For example, integrating latency as a critical parameter in the reward function has been shown to effectively train DRL agents for latency-sensitive service provisioning, allowing them to prioritize the requirements of time-critical applications. By aligning the reward function with the network’s performance goals, the agent can consistently learn policies that optimize the desired outcomes. REFERENCES [1] S. Cruzes, “Failure Management Overivew in Optical Networks,” IEEE Access, vol. 12, pp. 169 170–169 193, 2024. [2] S. Nallaperuma et al., “Interpreting Multi-Objective Reinforcement Learning for Routing and Wavelength Assignment in Optical Networks,” J. of Optical Commun. amd Netw., vol. 15, no. 8, pp. 497–506, 2023. [3] X. Chen et al., “DeepRMSA: A Deep Reinforcement Learning Framework for Routing, Modulation and Spectrum Assignment in Elastic Optical Networks,” J. Lightwave Technol., vol. 37, no. 16, pp. 4155 – 4163, 2019. [4] A. Beghelli et al., “Approaches to dynamic provisioning in multi-band elastic optical networks,” in 27th International Conference on Optical Network Design and Modelling (ONDM), 2023. [5] C. Natalino et al., “Optical Networking Gym: an Open-Source Toolkit for Resource Assignment Problems in Optical Networks,” J. of Optical Commun. amd Netw., vol. 16, no. 12, pp. G40–G51, 2024. [6] T. Tanaka and M. Shimoda, “Preand Post-Processing Techniques fpr Reinforcement-Learning-based Routing and Spectrum Assignment in Elastic Optical Networks,” J. of Optical Commun. amd Netw., vol. 15, no. 12, pp. 1018 – 1029, 2023. [7] A. Asiri and B. Wang, “Deep Reinforcement Learning for QoT-Aware Routing, Modulation, and Spectrum Assignment in Elastic Optical Networks,” J. of Lightwave Technologies, 2024. [8] E. Etezadi et al., “Deep Reinforcement Learning for Proactive Specrum Defragmentation in Elastic Optical Networks,” J. of Optical Commun. amd Netw., vol. 15, no. 10, pp. E86–E96, 2023. [9] C. Hern´ andez-Chulde et al., “Experimental Evaluation of a Latencyaware Routing and Spectrum Assignment Mechanism based on Deep Reinforcement Learning,” J. of Optical Commun. amd Netw., vol. 15, no. 11, pp. 925–937, 2023. [10] O. N. Foundation, “Transport api reference implementation agreement 1.1,” TR-547, 2022. [Online]. Available: https://opennetworking.org/wp-content/uploads/2020/08/TR-547TAPI-v2.1.3-Reference-Implementation-Agreement-1.pdf [11] C. Hern´ andez-Chulde et al., “Assessment of a latency-aware routing and spectrum assignment mechanism based on deep reinforcement learning,” in 2021 European Conference on Optical Communications (ECOC), 2021, pp. 1–4. [12] F. Musumeci et al., “A tutorial on machine learning for failure management in optical networks,” Journal of Lightwave Technology, vol. 37, no. 16, pp. 4125–4139, 2019. [13] C. Hern´ andez-Chulde et al., “Evaluation of deep reinforcement learning for restoration in optical networks,” in 2022 Optical Fiber Communications Conference and Exhibition (OFC), 2022, pp. 1–3. [14] R. Mart´ ınez et al., “Autonomous sdn-based global concurrent restoration for high-capacity optical metro networks,” in 2021 Optical Fiber Communications Conference and Exhibition (OFC), 2021, pp. 1–3. [15] ——, “Enhancing network performance and reducing power consumption in elastic optical networks with deep reinforcement learning,” in 28th International Conference on Optical Network Design and Modelling (ONDM), 2024. [16] Y. Xiong et al., “Lightpath Management in SDN-Based Elastic Optical Networks With Power Consumption Considerations,” J. Lightwave Technol., vol. 36, no. 9, pp. 1650 – 1660, 2018. [17] R. Mart´ ınez et al., “Optimizing Energy Efficiency in Optical Transport Networks through Autonomous AI-Assisted Control Operations,” in 2025 Optical Fiber Communications Conference and Exhibition (OFC), 2025, pp. 1–3. [18] C. Hern´ andez-Chulde et al., “DRL for VNF placement in Inter-Data Center Elastic Optical Networks,” in 2023 Optical Fiber Communications Conference and Exhibition (OFC), 2023, pp. 1–3. [19] ——, “Vnf placement over autonomic elastic optical network via deep reinforcement learning,” in ICC 2023 - IEEE International Conference on Communications, 2023, pp. 422–427. [20] ——, “Exploiting gnn and drl for online service provisioning over elastic optical networks,” in 2024 International Conference on Optical Network Design and Modeling (ONDM), 2024, pp. 1–3.