GMSA: A Digital Twin Application for Maritime Route and Event Forecasting

GMSA: A Digital Twin Application for Maritime Route and Event Forecasting Georgios Grigoropoulos ggrigoropoulos@kpler. com Kpler, Research Labs Athens, Greece Giannis Spiliopoulos [email protected] Kpler, Research Labs Athens, Greece Ilias Chamatidis [email protected] Kpler, Research Labs Athens, Greece Manolis Kaliorakis [email protected] Kpler, Research Labs Athens, Greece Alexandros Troupiotis - Kapeliaris [email protected] Kpler, Research Labs Athens, Greece University of the Aegean Ermoupolis, Syros, Greece Marios Vodas [email protected] Kpler, Research Labs Athens, Greece Evangelia Filippou [email protected] Kpler, Research Labs Athens, Greece Eva Chondrodima [email protected] University of Piraeus Piraeus, Greece Nikos Pelekis [email protected] University of Piraeus Piraeus, Greece Yannis Theodoridis [email protected] University of Piraeus Piraeus, Greece Dimitris Zissis [email protected] University of the Aegean Ermoupolis, Syros, Greece Konstantina Bereta [email protected] Kpler, Research Labs Athens, Greece ABSTRACT Digital twins are increasingly valuable in sectors like maritime, energy, logistics and transportation. In the maritime industry, the complexity of monitoring vessel traffic, necessitates more sophisticated, data-driven approaches due to the high volume of vessels and intricate movement patterns. This paper introduces GMSA, a digital twin application for maritime route and event forecasting for the entire global fleet, utilizing the real-time AIS streaming service of Kpler (MarineTraffic), for maritime event detection, vessel route prediction and traffic state estimation. Through the combined views of the real-time event detection functions, the vesseland port-specific data driven models, and the visualization of historical aggregated vessel mobility metrics, the application creates a a multi-layer information system for efficient, proactive action planning and enhanced decision making for the end-user. 1 INTRODUCTION Digital twins are virtual representations of physical, real-time environments [ 11 ]. They find applications in various fields like transportation, manufacturing, health, and energy, allowing for simulation, data collection, analysis, informed decision-making, and performance improvement. In maritime operations, digital twins could revolutionize monitoring, prediction, simulation, and route planning, enhancing decision-making and safety. Most current vessel traffic management and maritime monitoring solutions are designed for specific aspects of maritime operations, emphasizing areas like security, safety [ 10 , 15 , 18 , 20 ] and operations [ 1 , 3 , 8 , 11 , 13 , 16 ]. These solutions are typically deployed at a local or regional level based on specific use case ©2024 Copyright held by the owner/author(s). Published in Proceedings of the 27th International Conference on Extending Database Technology (EDBT), 25th March28th March, 2024, ISBN 978-3-89318-095-0 on OpenProceedings.org. Distribution of this paper is permitted under the terms of the Creative Commons license CC-bync-nd 4.0. limitations and requirements, focusing on monitoring traffic and event detection in predetermined sea areas such as coastal zones and ports. They primarily use the Automatic Identification System (AIS) for collecting vessel positions [ 14 ] and rely on basic models, leading to low accuracy and unreliability in critical situations. Their components primarily focus on delivering present situational information, neglecting the potential of historical and real-time data flows for automated event detection, forecasting, and action planning. To enhance capabilities, there’s a need for the fusion of historical insights with real-time, data-driven models to enable automated early event detection and forecasting for proactive action planning, improved interpretability, and reduced complexity for human operators. Kpler leverages a network of 6,200+ AIS receivers to collect information from vessels equipped with AIS transponders. Processing over 10 9 AIS messages daily, the MarineTraffic Service by Kpler provides real-time vessel tracking globally through its website and apps. The widespread coverage and volume of AIS messages demand robust algorithms and infrastructure for efficient data management, storage, retrieval, scalability, efficiency, and big data processing capabilities. This paper presents a novel digital twin application for maritime route and event forecasting: GMSA (Global Maritime Situational Awareness). The application targets the needs of key maritime user groups such as vessel crews and vessel traffic management operators. The main contributions of the application in the field of vessel traffic management systems include: (1) The application leverages a distributed system architecture that scales on a global level for real-time vessel traffic monitoring and maritime event forecasting based on the actor model [ 4 ]. This addresses key shortcomings in scalability, scope, speed and accuracy of present vessel traffic management solutions through the fusion of heterogeneous Demonstration Paper Series ISSN: 2367-2005 798 10.48786/edbt.2024.73 extreme-scale data associated with diverse layers of information that are specific to individual vessels, delivering a digital twin of the global fleet. (2) The application includes a variety of functions found in current vessel traffic management and monitoring systems for data exploration, filtering and event detection. Additionally, it expands the capabilities of vessel traffic management systems through data-driven vessel route and event forecasting models and functions, integrated on an actor level, which leverage vessel-specific information for real-time global vessel traffic monitoring, route forecasting and precise maritime event predictions. The User Interface (UI) visualizes the current vessel traffic in an area from streaming real-time AIS data transmitted by the global fleet which is captured by satellite services, the AIS terrestrial network of Kpler and third-party providers of AIS real-time data operating globally. 2 SYSTEM ARCHITECTURE The system architecture, presented in Figure 2, addresses the scalability and forecasting limitations of existing maritime situational awareness systems through the adoption of a scalable, distributed architecture that fuses extreme-scale data related to vessels, sea areas, and routes. Data-driven models, trained on historical vessel mobility data, are integrated, along with realtime AIS data, event detection functions, and visualization of historical vessel mobility metrics. This creates a multi-layered information system for efficient action planning and enhanced decision-making. To address scalability, real-time processing, and forecasting requirements, the system adopts a multi-layered approach based on the actor model [ 4 ], as implemented by the Akka framework [ 5 ]. The system utilizes lightweight, isolated actors for scalability and responsiveness, while incorporating mechanisms for handling state and failures. Real-time processing involves the ingestion of data streams from multiple Kafka connections [ 7 ], including the AIS global terrestrial network by MarineTraffic (Kpler), AIS satellite services, and third-party AIS data providers globally. Additionally, the system partitions the data stream to leverage the actor model, generating multiple actors corresponding to specific vessels defined by unique Maritime Mobile Service Identity (MMSI). To enhance forecasting capabilities, the system integrates a novel short-term route forecasting model at the actor level. The system utilizes a 1-to-1 mapping for each vessel actor, ensuring dedicated predictions per vessel in alignment with the digital twin concept. The spatial aspect involves defining two additional actor classes using the H3 spatial index [ 17 ]: one for proximity event detection and another for collision forecasting. These actors process the combined output of all vessel actors and communicate the state of their respective event classes back to affected vessel subsets. Finally, actor states are stored in a Redis database [ 9 ] by a writer actor, enabling visualization through a dedicated API in the Middleware component. This facilitates end-user interaction through the UI, allowing exploration of visualized routes and event states. 3 LONG-TERM VESSEL ROUTE FORECASTING The system incorporates EnvClus* [ 19 , 22 ] for long-term forecasting (L-VRF) by making API calls to predict vessel paths from a specific origin to a destination port. EnvClus* utilizes historical AIS data from MarineTraffic (Kpler) to train dedicated models for each origin-destination port pair, clustering positional AIS data to extract common vessel pathways (Figure 1iii). EnvClus* scales globally for various origin-destination pairs and vessel types, providing aggregated mobility statistics called Patterns of Life [ 21 ] for a comprehensive historical traffic overview (Figure 1iv). The fusion of present vessel positions, route forecasts, and aggregated mobility insights enables users to assess route efficiency, explore rerouting strategies, and identify deviations from common traffic patterns. 4 SHORT-TERM VESSEL ROUTE FORECASTING In [ 2 ], a state-of-the art Short-term Vessel Route Forecasting (S-VRF) Long Short Term Memory (LSTM) model is introduced for predicting vessel trajectories using past AIS positions. The model considers AIS database characteristics, including irregular AIS transmissions, and generates predictions based on variable input sequences of past vessel displacements. To meet system requirements and enhance efficiency, a new S-VRF model architecture is defined, reducing tensor input and output size to fixed dimensions, and adapting to irregularities and limitations of streaming AIS transmissions. The model architecture includes a bidirectional LSTM layer with in-layer regularization to reduce overfitting. The model trained and tested using archived AIS stream data from MarineTraffic (Kpler), demonstrates superior performance compared to linear kinematic models and is integrated with the system architecture at an actor level. Trajectory predictions are visualized for end-users through the UI (Figure 1v). 5 VESSEL TRAFFIC FLOW FORECASTING The main objective of Vessel Traffic Flow Forecasting (VTFF) is to predict future vessel traffic in a specific region based on historical traffic flow data represented as spatiotemporal raster data. VTFF, methods mainly utilize grid-based representation analysis, approaching the problem indirectly through route forecasting or directly as a flow sequence forecasting problem. Comparative analysis in [ 12 ] shows that the indirect VTFF strategy, particularly when integrated with the S-VRF model, demonstrates superior prediction accuracy and lower computational demands compared to direct strategies. The VTFF approach, utilizing the S-VRF model, is integrated into system, processing streaming AIS messages to generate future trajectories. The S-VRF model predicts vessel locations at a sampling rate of 5 minutes to 30 minutes, allocating them to the H3 spatiotemporal grid [ 17 ], visualized in Figure 1vi, where vessel counts in each grid cell represent the vessel traffic flow. 6 EVENT DETECTION AND FORECASTING The system offers standard maritime event detection functions, capturing events related to vessel movement and AIS transmission status. Examples of such events encompass the deactivation of the AIS transmitter on a vessel [ 6 ] (Figure 1vii) and close proximity between vessels (Figure 1viii). Additionally, the system 799 Figure 1: Visualization of the key features and forecasting components on the user interface. i) Vessel information inspection, ii) Vessel type filtering, iii) L-VRF, iv) Aggregated vessel mobility statistics inspection in L-VRF, v) S-VRF, vi) VTFF, vii) AIS-switch off event detection, viii) Proximity event detection, ix) Vessel collision forecasting using the S-VRF. Figure 2: The GMSA system architecture based on the actor model. integrates a vessel collision forecasting algorithm at the actor level, utilizing the S-VRF model to detect potential collisions between vessels. Each vessel’s AIS message generates a route forecasting prediction with the S-VRF model, resulting in seven positions per message. The algorithm assesses temporal and spatial intersections of forecasted trajectories within the 30-minute prediction window, logging potential collisions for the end-user if both conditions are met. The user can receive notifications, 800 view collision details and check estimated collision times and involved vessels through the UI, as illustrated in Figure 1ix. 7 DEMONSTRATION PLAN The demonstration aims to showcase the main models and functions through a video with guided use case scenarios (view the demo video here). First, the demonstration presents the UI and instructions for vessel information inspection and filtering. Subsequently, the main models and event detection and forecasting functions are presented. Users aiming to use the service for long-term route forecasting and planning will be able to extract the routes for vessels arriving at the ports of Algeciras in Spain and Piraeus in Greece using the EnvCLUS* model. Along the forecasted routes, the derived historical vessel mobility insights are visualized as a heatmap through the H3 index [ 17 ]. The combination of the EnvCLUS* model with the mobility patterns allows users to identify the most common route(s) as well as alternative routes towards the port of interest, observe deviations from the typical routes and analyze historical traffic patterns along route corridors. Users that want to utilize the service for forecasting vessel movements in the short-term can choose the preferred vessel route forecasting model and utilize the vessel traffic flow forecasting function for enhanced vessel traffic monitoring, planning, and situational awareness. Finally, users can use the service for maritime event detection and forecasting. Maritime event classes are detected and forecasted for all vessels tracked on the AIS MarineTraffic live stream. Event lists are generated depending on the specific event detection in the sea area of interest. AIS switch-off events and the proximity event detection between two vessels are supported. By leveraging the short-term route forecasting models, users may also detect and visualize collision event forecasts in the live view. 8 CONCLUSIONS This paper presents GMSA, a scalable digital twin application for maritime route and event forecasting that utilizes the realtime MarineTraffic AIS streaming service by Kpler, for maritime event detection, vessel route prediction and traffic state estimation. GMSA addresses key limitations of current vessel traffic management systems regarding their scalability and forecasting capabilities. In future work we aim to uphold and enhance the application’s highly scalable and adaptable architecture and develop a comprehensive representation of global vessel mobility, serving as a digital twin for Global Maritime Situational Awareness. ACKNOWLEDGMENTS The presented application (GMSA), the system architecture, the Short-term (S-VRF) and Long-term (EnvClus*) Vessel Route Forecasting model versions, the version of the indirect Vessel Traffic Flow Forecasting methodology adopted from [ 12 ] as implemented within the proposed system architecture in this paper and the event detection and forecasting functions, were developed within this work and are owned by Kpler. This research is partially supported by European Union’s Horizon 2020 research and innovation programme under grant agreement No 957237, project Enabling Maritime Digitalization by Extreme-scale Analytics, AI and Digital Twins (VesselAI) and by European Union’s Horizon 2020 research and innovation programme under grant agreement No. 101092749, project “Critical Action Planning over Extreme-Scale Data (CREXDATA)”. REFERENCES [1] Nur Assani, Petar Matić, and Marko Katalinić. 2022. Ship’s Digital Twin—A Review of Modelling Challenges and Applications. Issue 12. https://doi.org/ 10.3390/app12126039 [2] Eva Chondrodima, Nikos Pelekis, Aggelos Pikrakis, and Yannis Theodoridis. 2023. An Efficient LSTM Neural Network-Based Framework for Vessel Location Forecasting. IEEE Transactions on Intelligent Transportation Systems 24, 5 (2023), 4872–4888. https://doi.org/10.1109/TITS.2023.3247993 [3] Samantha Cope, Ellen Hines, Roger Bland, Jerry D. Davis, Brendan Tougher, and Virgil Zetterlind. 2020. Application of a New Shore-Based Vessel Traffic Monitoring System Within San Francisco Bay. Frontiers in Marine Science 7 (2 2020). https://doi.org/10.3389/fmars.2020.00086 [4] Carl Hewitt, Peter Bishop, and Richard Steiger. 1973. A universal modular actor formalism for artificial intelligence. In Proceedings of the 3rd international joint conference on Artificial intelligence. 235–245. https://doi.org/10.5555/ 1624775.1624804 [5] Lightbend Inc. 2022. Akka. http://akka.io/ [6] Ioannis Kontopoulos, Konstantinos Chatzikokolakis, Dimitris Zissis, Konstantinos Tserpes, and Giannis Spiliopoulos. 2020. Real-time maritime anomaly detection: detecting intentional AIS switch-off. International Journal of Big Data Intelligence 7, 2 (2020), 85–96. https://doi.org/10.1504/IJBDI.2020.107375 [7] Jay Kreps. 2011. Kafka : a Distributed Messaging System for Log Processing. https://api.semanticscholar.org/CorpusID:18534081 [8] Jae-Hoon Lee, Yoon-Seo Nam, Yonghwan Kim, Yuming Liu, Jaehak Lee, and Heesuk Yang. 2022. Real-time digital twin for ship operation in waves. Ocean Engineering 266 (2022), 112867. https://doi.org/10.1016/j.oceaneng. 2022.112867 [9] Redis Ltd. 2023. Redis. https://redis.io/ [10] ST Engineering Advanced Networks & Sensors Pte Ltd. 2021. Maritime Situational Awareness System. [11] Zhihan Lv, Haibin Lv, and Mikael Fridenfalk. 2023. Digital Twins in the Marine Industry. Electronics 12 (2023). Issue 9. https://doi.org/10.3390/ electronics12092025 [12] Petros Mandalis, Eva Chondrodima, Yannis Kontoulis, Nikos Pelekis, and Yannis Theodoridis. 2022. Machine Learning Models for Vessel Traffic Flow Forecasting: An Experimental Comparison. In 2022 23rd IEEE International Conference on Mobile Data Management (MDM). 431–436. https://doi.org/10. 1109/MDM55031.2022.00094 [13] Kongsberg Maritime. 2023. Situational Awareness - Enhancing safety and efficiency. [14] International Maritime Organization. 1980. SOLAS Chapter V, Regulation 19.2. https://www.navcen.uscg.gov/sites/default/files/pdf/AIS/SOLAS.V.19.2. 1-5.pdf [Accessed 16-10-2023]. [15] Science, Technology Organisation Centre for Maritime Research, and Experimentation (STO CMRE). 2023. Maritime Security - Maritime Situational Awareness (MSA). [16] Panagiotis Tampakis, Eva Chondrodima, Aggelos Pikrakis, Yannis Theodoridis, Kostis Pristouris, Harry Nakos, Eleni Petra, Theodore Dalamagas, Andreas Kandiros, Georgios Markakis, Irida Maina, and Stefanos Kavadas. 2020. Sea Area Monitoring and Analysis of Fishing Vessels Activity: The i4sea Big Data Platform. (2020), 275–280. https://doi.org/10.1109/MDM48529.2020.00063 [17] Uber Technologies. 2019. Uber H3: Uber’s Hexagonal Hierarchical Spatial Index. [18] Terma. 2023. Complete Situational Awareness. [19] Alexandros Troupiotis-Kapeliaris, Nicolas Zygouras, Manolis Kaliorakis, Spiros Mouzakitis, Giannis Tsapelas, Alexander Artikis, Eva Chondrodima, Yannis Theodoridis, and Dimitris Zissis. 2022. Data Driven Digital Twins for the Maritime Domain. In Technology and Science for the Ships of the Future. IOS Press, 744–751. https://doi.org/10.3233/PMST220087 [20] A. C. van den Broek, Martin Neef, Patrick Hanckmann, Sicco Pier van Gosliga, and D. van Halsema. 2011. Improving maritime situational awareness by fusing sensor information and intelligence. 14th International Conference on Information Fusion (2011), 1–8. https://api.semanticscholar.org/CorpusID: 2371319 [21] Dimitris Zissis, Konstantinos Chatzikokolakis, Giannis Spiliopoulos, and Marios Vodas. 2020. A Distributed Spatial Method for Modeling Maritime Routes. IEEE Access 8 (2020), 47556–47568. https://doi.org/10.1109/ACCESS.2020. 2979612 [22] Nikolas Zygouras, Alexandros Troupiotis-Kapeliaris, and Dimitris Zissis. 2024. EnvClus*: Extracting Common Pathways for Effective Vessel Trajectory Forecasting. IEEE Access 12 (2024), 3860–3873. https://doi.org/10.1109/ACCESS. 2023.3349081 801