GN5-2 WFO Telemetry Module Incubator Project: Closure Report

© GÉANT Association on behalf of the GN5-1 project. The research leading to these results has received funding from the European Union’s Horizon Europe research and innovation programme under Grant Agreement No. 101100680 (GN5-1). Co-funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them. 10-12-2024 WFO Telemetry Module Incubator Project: Closure Report Grant Agreement No.: 101100680 Work Package: 6 Task Item: 4 Dissemination Level: PU (Public) Document ID: GN5-1-24-KI4VET Authors: Peter Boers (SURF), Ivana Golub (PSNC) Abstract The WFO Telemetry Module was created to enable High Resolution data streaming from the networks. This incubator project resulted in a Helm Chart that enables users to deploy a highly scalable architecture of software on Kubernetes, an example implementation, and a GÉANT infoshare. WFO Telemetry Module Incubator Project: Closure Report Document ID: GN5-1-24-KI4VET ii Table of Contents Executive Summary 1 1 Introduction 2 2 Project Results 3 3 Next Steps and Future Work 4 4 Conclusions 4 References 5 Glossary 6 Table of Figures Figure 1.1: Architecture of an AIOps-enabled network with WFO and the gNMIc telemetry platform 3 WFO Telemetry Module Incubator Project: Closure Report Document ID: GN5-1-24-KI4VET 1 Executive Summary The WFO Telemetry Module incubator project, run under the Network Development Work Package (WP6) [NETDEV] Incubator [NETDEV-Inc] of the GN5-1 project, successfully developed a scalable, vendor-agnostic telemetry solution for network operations. The project focused on deploying the gNMIc telemetry tool on Kubernetes, integrating it with the Workflow Orchestrator, and enabling advanced monitoring and operational insights for networks. Key achievements include: • Creation of a Helm chart for deploying gNMIc on Kubernetes, promoting dynamic scalability and realtime data processing. • Integration with the Workflow Orchestrator to automate network target discovery and streamline telemetry workflows. • Development of a proof-of-concept interface for unified network resource visualisation. • Dissemination of results through GitHub, NREN channels, and a GÉANT info-share event. The project has no major security vulnerabilities, as confirmed by an independent scan. By leveraging this solution, NRENs and networking organisations can implement their own high-resolution telemetry infrastructure, driving innovation in network management and Artificial Intelligence for IT Operations (AIOps) integration. WFO Telemetry Module Incubator Project: Closure Report Document ID: GN5-1-24-KI4VET 2 1 Introduction Rationale A vendor-agnostic telemetry platform is needed to facilitate the introduction of AIOps in an orchestrated multivendor network and is a pre-requisite for introducing Machine Learning into Network Operations. The work on developing a deployment setup using open-source tooling that is highly scalable and capable of streaming telemetry from gNMI-capable network devices extends the current work of WP7 T3 and provide a pluggable module for the Workflow Orchestrator (WFO). Architecture and Tool Choice There are a number of tools/plugins that support the use of gNMI as a method of streaming telemetry. However, they make use of static configuration, are not designed to scale dynamically, and are only able to query metrics that are configured through a configuration file. gNMIc is a tool that is designed with scalability and flexibility out of the box and it also provides a gRPC interface to interact with the network on demand. As networks and network services change, it is necessary to be able to dynamically add or reduce the number of targets that are monitored and scale up or down according to the number of metrics that are gathered. Deploying gNMIc on Kubernetes and combining cloud native technology with an event-based architecture enables the telemetry solution to process real-time events from the network and be able to feed a data pipeline that can be used for machine learning algorithms or provide operators with real time metrics in tools such as the Workflow Orchestrator Graphical User Interface (GUI) and/or Grafana. As well as providing telemetry data, gNMIc can also be used to command the network to provide, on demand, operational data such as optic power levels, BGP status, interface status, displaying of routing/mac tables by using the gNMI protocol to communicate with the network devices. This makes it is possible to integrate operational device data into the Workflow Orchestrator GUI, so Network Operations Centre (NOC) operators can combine customer facing information with resource facing information in a unified view. During the course of this project, the Openconfig project (which develops gNMIc) did not provide resources to deploy gNMIc on Kubernetes. Therefore, this project developed these resources so that an autoscaling gNMIc pipeline can be deployed. Furthermore, this project created an integration in the Workflow Orchestrator that enables gNMIc to automatically discover network targets to stream data from. This integration is optional in its usage; that is, the user does not have to deploy the Workflow Orchestrator in combination with the gNMIc deployment to make use of the proposed telemetry solution. Lastly, this proposal created a proof-of-concept implementation in the Workflow Orchestrator GUI that shows the Admin and Operational status of network interfaces per ‘reference node product’. This work explores how the gNMI interface can be used to show operational, resource facing telemetry, together with customer-facing resources in a Network Management System style pane of view in the Workflow Orchestrator GUI. The architecture of an AIOps-enabled network making use of gNMIc as a telemetry platform and the Workflow Orchestrator as network orchestrator is drawn in Figure 1 below. WFO Telemetry Module Incubator Project: Closure Report Document ID: GN5-1-24-KI4VET 3 Figure 1.1: Architecture of an AIOps-enabled network with WFO and the gNMIc telemetry platform The areas highlighted in purple are a part of the output of this project. The work in this project was supported by GÉANT and HEAnet. 2 Project Results This project has a number of results: • Helm chart hosted here: https://github.com/workfloworchestrator/gnmic-cluster-chart • This was promoted on various NRENand other network-related channels on Discord and Slack, in particular the NREN Slack and WFO Discord. • An example implementation of this technology stack in the workflow orchestrator. More information about this can be found by emailing the Workflow Orchestrator programme board at [email protected] • GEANT info share. This event showcased the output of this project. The presentation and recording are archived here: https://events.geant.org/event/1752/ • Security report and scan executed in WP9 T2. The main conclusion of this report is that there are no major vulnerabilities in the software. The main sections of a document are where the subject of the report is discussed in detail in a structured way, for example setting out the research conducted and methodology used, the description and analysis of the main work carried out, and any findings/results and subsequent actions. WFO Telemetry Module Incubator Project: Closure Report Document ID: GN5-1-24-KI4VET 4 3 Next Steps and Future Work The next steps for this work would be to: • Increase the level of documentation by incorporating some implementation examples in the docs. • Incorporate the recommendations of the Security scan and report in a new release of the module. There are no major vulnerabilities that need to be addressed. • Publish the cluster chart through nmaas: https://nmaas.geant.org 4 Conclusions This document reports the successful execution of the WFO telemetry module incubator project. During the past 10 months, SURF implemented and demonstrated a highly scalable streaming telemetry setup for a vendor agnostic network. By using the output of this project, NRENs and other networking organisations can deploy their own streaming telemetry infrastructure. This work has been published online on GitHub and introduced to the wider NREN and European community through an info share and various posts on commonly used Slack and Discord channels. WFO Telemetry Module Incubator Project: Closure Report Document ID: GN5-1-24-KI4VET 5 References [NETDEV] https://wiki.geant.org/display/NETDEV/ [NETDEV-inc] https://wiki.geant.org/display/NETDEV/NETDEV+Incubator WFO Telemetry Module Incubator Project: Closure Report Document ID: GN5-1-24-KI4VET 6 Glossary AIOps Artificial Intelligence for IT Operations GN5-1 GÉANT Network 5 Phase 1 project, part-funded from the EU’s Horizon Europe research and innovation programme under Grant Agreement No. 101100680 gNMI gRPC Network Management Interface gNMIc gRPC Network Management Interface (client) gRPC Google Remote Procedure Calls GUI Graphical User Interface NREN National Research and Education Network NOC Network Operations Centre WFO Workflow Orchestrator WP Work Package WP6 Work Package 6 (Network Development) WP7 T3 Work Package 7 (Network Core Infrastructure and Core Service Evolution and Operations) ; Task 3 (Network Management, Automation and Orchestration) WP9 T2 Work Package 9 (Operations Support) ; Task 2 (Software Governance and Support)