Towards Knowledge-Driven Task Planning for Contextual Autonomy in Rescue Scenarios

Towards Knowledge-Driven Task Planning for Contextual Autonomy in Rescue Scenarios 1st Alessandro Umbrico CNR-ISTC Italy [email protected] 2nd Francesca Fracasso CNR-ISTC Italy [email protected] 3rd Andrea Orlandini CNR-ISTC Italy [email protected] Abstract—The complexity of natural disaster incidents demands innovative technological solutions to support first responders in their efforts. The increasing autonomy of UGVs and UAVs, along with the improvement in perception capabilities in terms of both the quantity and accuracy of data, necessitate the design of advanced technologies that can contextually frame and abstract data to aid first responders in making informed decisions. In this context, we investigate the design of a cognitive framework to support mission-level decisions and coordinate the autonomy of the UGV by integrating knowledge representation with task planning. On the one hand, semantic technologies aggregate perception data collected from available sources and support situation awareness by inferring relevant situations and suitable mission-level tasks that could be performed. On the other hand, task planning contextually coordinates the behaviors of UGVs to accomplish mission-level goals and support first responders. Index Terms—Situation Awareness, Task Planning, Knowledge Representation and Reasoning I. INTRODUCTION The management of natural disasters presents high-stakes decisions that place humans under high pressure to quickly evaluate a large amount of information and rationally assess the consequences of possible choices. Modern robotic devices and advanced computational capabilities offered by Artificial Intelligence (AI) could support the decisional process of first responders (FRs). Unmanned Ground Vehicles (UGVs) and Unmanned Aerial Vehicles (UAVs) can improve situational awareness through their perception. Acquired raw data is typically processed by a central ground station and offered to FRs for an updated and rich visualization of the scene. Within the Ground-Station, AI-based techniques are employed to process raw data (e.g., images) to detect types and states of objects. However, having more information does not automatically imply better decisions. Especially in critical scenarios, it is crucial to adapt the stimuli according to the operational context and emotional response of human users [1]. The large volume of data and the high number of possible choices risk cognitive overloading of FRs. In this context, the European project TRIFFID 1[2] investigates the design of a comprehensive technical framework capable of integrating UGV and UAV with AI-based functionalities to improve disaster response. Authors are partially funded by the EU’s Horizon Europe research and innovation programme under grant agreement No 101168042 (TRIFFID). 1https://triffid-project.eu This paper introduces our efforts regarding the integration of knowledge representation and reasoning technologies with automated planning to support the situation awareness and contextual autonomy of UGVs within rescue missions. We investigate the use of ontology-based semantic technologies to aggregate tagged data received from the environment, detect relevant situations/events, and automatically infer suitable tasks that can be performed. Automated task planning is then used to coordinate mission resources. II. KNOWLEDGE-DRIVEN REASONING AND PLANNING The TRIFFID system aims to enrich the services of the Base of Operation (BoO) typically set by FRs in proximity to the disaster site. The system consists of a ground station, a tele-operated UAV, and an autonomous UGV. The UGV accompanies the field crew within the disaster scene. The ground station provides the local Disaster Scene Commander (DCS) with a unified and updated view of the disaster scene, based on data collected during operations. The ground station is initialized with Earth Observation (EO) data that are analyzed to construct a semantically annotated map of the area of interest. The UAV is deployed to acquire additional data about specific areas or individual entities. The ground station operation makes strategic decisions by coordinating and setting mission-level tasks that are dispatched to selected resources (UGV or field crew members). The ground station centralized the AI services necessary to monitor and coordinate the execution of a mission. Such services help DCS maintain an informed and synchronized view of a mission and make timely and accurate decisions. Our contribution focuses on decision-making and missionlevel control of UGVs. We propose the use of knowledgebased representation and reasoning to interpret environmental data and reason about mission-level tasks to be performed [3]. Timeline-based planning [4] then “operationalizes” operations by automating task decomposition and coordinating the UGV and field crew members. A key aspect of TRIFFID is the capability to integrate the large amount of sensory data and images produced through satellite observations (EO), UGV, and UAV perception. We aim to realize situation awareness processes through ontological semantics. Figure 1 depicts the conceptual abstraction process as introduced in [5]. We extend an ontology designed 2025 I-RIM Conference October 17-19, Rome, Italy ISBN: 9788894580570 10.5281/zenodo.17629754 125 Fig. 1. Situation Awareness abstraction process [5] Fig. 2. Task Planning integration concepts for situation awareness [5] with a semantic model of robot tasks [6] to detect critical situations and contextually reason about possible tasks to be executed. On the one hand, we use ontological models about sensor data [7] to aggregate perception data coming from the environment. We then define a rescue domain ontology to semantically characterize mission contexts (e.g., preparation, monitoring, intervention) together with situations and events that could be detected (e.g., smoke, human body found, building infrastructural conditions). Reasoning mechanisms would, in particular, contextualize inferred situations according to the types of entities involved and assign priorities or risk levels to them. For example, fire or smoke detected within a building known to be a school would receive higher priority than the same situation detected in an anonymous and abandoned building. The envisaged abstraction process is crucial to not overload DCS with information that could limit its evaluation of operational needs. Semantics would frame information and help DCS maintain the focus on specific emerging needs, taking advantage of the enhanced perception functionalities of the technology. On the other hand, mission-level tasks composing operational procedures (e.g,, exploration, inspection, rescue, monitoring) would be contextualized according to the current context of a mission and the set of (prioritized) situations that have been detected. The outcome of the reasoning would provide the DCS with a list of prioritized situations and recommended tasks to be performed in the current context of the mission. In combination with situation awareness, task planning keeps track of the execution of a mission and coordinates the implementation of mission-level tasks. It receives mission-level task requests from DCS and assigns operations to UGV and field crew members, according to their current state within the mission. Robot navigation for indoor and outdoor autonomy is guaranteed through state-of-the-art control [8]. Figure 2 shows the high-level architecture supporting the integration of task planning and motion planning through an MQTT broker. We enrich robot awareness through integrated task and motion planning to dynamically set the robot’s goal and navigation modalities according to the mission needs [9]. The navigation and interaction behavior of the UGV is thus set according to the requirements of the task in execution. The motion planner of the navigation module would indeed expose a set of navigation modalities (e.g., environment mapping, search, approach, follow-me). The task planner thus adapts the navigation behavior of UGV to the contextual objectives of the mission-level task. III. CONCLUSIONS This paper introduces the main concepts of the TRIFFID system, designed to support first responders in disaster management scenarios. The contribution specifically concerns the integration of ontology-based reasoning and task planning to support situation awareness and contextual control of robot navigation behaviors. Next steps will focus on the prototyping of the integrated task and motion planning for rescue, and the design of an ontological model for contextual abstraction of environment data. REFERENCES [1] A. Cesta, G. Cortellessa, and R. De Benedictis, “Training for crisis decision making – An approach based on plan adaptation,” KnowledgeBased Systems, vol. 58, pp. 98–112, 2014. [2] J. Cani, P. Koletsis, K. Foteinos, I. Kefaloukos, L. Argyriou, M. Falelakis, I. D. Pino, A. Santamaria-Navarro, M. ˇ Cech, O. Severa, A. Umbrico, F. Fracasso, A. A. Orlandini, D. Drakoulis, E. Markakis, I. Varlamis, and G. T. Papadopoulos, “TRIFFID: Autonomous Robotic Aid For Increasing First Responders Efficiency,” in 2025 6th International Conference in Electronic Engineering & Information Technology (EEITE), pp. 1–9, 2025. [3] A. Umbrico, A. Cesta, G. Cortellessa, and A. Orlandini, “A Holistic Approach to Behavior Adaptation for Socially Assistive Robots,” International Journal of Social Robotics, vol. 12, no. 3, 2020. [4] M. Cialdea Mayer, A. Orlandini, and A. Umbrico, “Planning and execution with flexible timelines: a formal account,” Acta Informatica, vol. 53, no. 6-8, 2016. [5] M. M. Kokar, C. J. Matheus, and K. Baclawski, “Ontology-based situation awareness,” Information Fusion, vol. 10, no. 1, pp. 83–98, 2009. [6] C. Schlenoff and E. Messina, “A robot ontology for urban search and rescue,” in Proceedings of the 2005 ACM Workshop on Research in Knowledge Representation for Autonomous Systems, KRAS ’05, (New York, NY, USA), pp. 27–34, Association for Computing Machinery, 2005. [7] K. Janowicz and M. Compton, “The Stimulus-sensor-observation Ontology Design Pattern and Its Integration into the Semantic Sensor Network Ontology,” in Proceedings of the 3rd International Conference on Semantic Sensor Networks - Volume 668, SSN’10, (Aachen, Germany, Germany), pp. 64–78, CEUR-WS.org, 2010. [8] A. Santamaria-Navarro, S. Hern´ andez, F. Herrero, A. L´ opez, I. d. Pino, N. Rodr´ ıguez-Linares, C. Fern´ andez, A. Bald´ o, C. Lemardel´ e, A. Garrell, J. Vallv´ e, H. Taher, A. M. Puig-Pey, L. Pag` es, and A. Sanfeliu, “Toward the Deployment of an Autonomous Last-Mile Delivery Robot in Urban Areas: The Ona Prototype Platform,” IEEE Robotics & Automation Magazine, vol. 32, no. 3, pp. 26–39, 2025. [9] P. Singamaneni, M. Tascioni, A. Umbrico, A. Orlandini, and R. Alami, “Contextual Social Navigation Through Integrated Task and Motion Planning,” 2025. 126