Evolution of Transferable and Self-Organized Communication Modules for Solving Multiple Swarm Robotics Tasks

IEEE TRANSACTIONS ON CYBERNETICS 1 Evolution of Transferable and Self-Organized Communication Modules for Solving Multiple Swarm Robotics Tasks Rafael Sendra-Arranz , Álvaro Gutiérrez ,Senior Member, IEEE, and Anders Lyhne Christensen ,Senior Member, IEEE Abstract—A key aspect of decentralized multirobot coordination is communication. However, beyond simple signaling, there are only few reports in the literature on the successful evolution of communication, with successes largely dependent on specific tasks and evolutionary setups. Thus, there is a lack of standardized communication frameworks that can be applied to different tasks without the need to redesign, rebuild, or re-evolve the entire system for every new task. In this article, we propose a novel communication module that does not need to be modified for its use in different tasks. Each robot has a coordinate (state) in a virtual communication space. The communication space is partitioned into virtual regions, and each region is linked to a physical behavior, such as seeking resources, phototaxis, or recharging the battery. A robot’s individual behavior is determined by the region to which its current communication state belongs. Since robots can navigate the communication space and continually broadcast their coordinates to neighbors within range, robot swarms can effectively coordinate their behavior in a self-organized manner. We demonstrate that the same evolved communication module is effective in three swarm robotics tasks: 1) the physical aggregation of the robots into groups of a desired size; 2) the formation of desired swarm geometries; and 3) a foraging task based on temporal role allocation. The results show that the communication module provides good and scalable performance in all tasks, representing a significant step toward a task-agnostic communication framework for robot swarms. Index Terms—Collaborative intelligence, communication systems, distributed management, evolutionary robotics, formation control, swarm robotics. I. INTRODUCTION COMMUNICATION is one of the key pillars in swarm robotics (SR) [1],[2] that is required to effectively Received 10 October 2024; revised 18 March 2025 and 1 September 2025; accepted 5 September 2025. This work was supported in part by the European Commission within the context of the project SMAUG, through EU Horizon Europe under Grant 101121129; in part by the MCIN/AEI/10.13039/501100011033 under Grant PID2023-146540OB-C42; and in part by the Independent Research Fund Denmark under Grant 0136-00251B. The work of Rafael Sendra-Arranz was supported by the “Programa Propio I+D+i” financed by the Universidad Politécnica de Madrid. This article was recommended by Associate Editor P. Shi. (Corresponding author: Rafael Sendra-Arranz.) Rafael Sendra-Arranz and Á. Gutiérrez are with the E.T.S. Ingenieros de Telecomunicación, Universidad Politécnica de Madrid, 28040 Madrid, Spain (e-mail: r[email protected]; [email protected]). Anders Lyhne Christensen is with the SDU UAS Center, MMMI, University of Southern Denmark, 5230 Odense, Denmark (e-mail: [email protected]). Color versions of one or more figures in this article are available at https://doi.org/10.1109/TCYB.2025.3610013. Digital Object Identifier 10.1109/TCYB.2025.3610013 solve many cooperative tasks (e.g., [3],[4], and [5]). Interrobot communication can potentially be any sort of information exchange among the robots of the swarm. In SR, communication can be categorized as either direct interaction [6],[7],[8],stigmergy [9],[10],[11],ordirect communication [3],[4],[5]. The third category, direct communication, requires explicit transmission and processing of signals whose meaning is correlated to the specific task. There are multiple works in the literature that employ direct communication in SR tasks, with diverse communication technologies and different emergent or designed communication semantics (e.g., [4],[5],[12],[13], [14], and [15]). Unfortunately, the communication mechanics and semantics in these SR works are specifically designed ad hoc for the task or problem studied [16]. This approach represents a significant challenge in the SR field, as there is a lack of standardized communication frameworks or methods that can be implemented across a wide variety of problems. This issue is also discussed in [17], highlighting the relation between the lack of standardization and realworld applicability in the SRs field. Therefore, designing and evolving SR communication systems and mechanics that are task agnostic is a key challenge in the field. In this article, we propose a self-organized and decentralized communication module whose logic is evolved only once and can be used in different SR tasks and with varying swarm sizes without modification. We demonstrate this valuable feature by employing the exact same evolved communication module in three well-known SR problems, namely, aggregation of the robots into groups of desired sizes, formation control, and a foraging task involving dynamic role allocation. We show that these distinct tasks can be successfully solved using the proposed system. The communication module is based on a virtual state space, referred to as the communication space. Each robot has a coordinate in this communication space that corresponds to its communication state. The communication space is partitioned into regions, each of which is associated with a physical behavior—for example, behaviors, such as “follow another robot” or “go to a light source.” Each robot executes the behavior associated with the region that contains its current communication state. Robots can dynamically change their communication state by virtually navigating in the communication space. Furthermore, at each control cycle, every robot c 2025 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 2IEEE TRANSACTIONS ON CYBERNETICS communicates its communication state to its neighbors within range. Robots can use the received states to coordinate their virtual navigation with other robots in order to solve SR tasks. For example, in a multiplace foraging task, we demonstrate how robots can use the communication module to coordinate and decide which robots forage from each food source. The transferability of robot behaviors has been addressed by multiple authors in the literature. Kegeleirs et al. [18] define two types of transferability: 1) design-method transfer and 2) embodiment transfer. The former refers to the ability to use the same automatic design method to generate robot controllers for different platforms or tasks, while the latter implies transferring and deploying the robot controller across multiple scenarios. In this article, we focus on embodiment transfer, slightly extending the definition in [18] toward communication transferability—the reuse of the same evolved communication across multiple missions. Hasselmann et al. [19] proposed an automatic and modular design method for robot swarms that minimizes human intervention. Probabilistic finite-state machines (PFSMs) were automatically built for different tasks using ANN-based, mission-agnostic behaviors. However, the tasks considered in this work did not require explicit communication and, therefore, communication transfer was not necessary. Salman et al. [20] proposed the automatic design of stigmergic communication and its application to a wide variety of tasks. Even though they achieved automatic communication design and its application to multiple tasks, the optimization of communication must still be restarted on a per-mission basis (design-method transfer). Previous studies have reported the use of virtual structures to coordinate robot swarms [21],[22],[23],[24],[25]. Nonetheless, these virtual constructs are used as tools for planning robot motion rather than as a means of communication. A related approach was proposed in [26] and extended in [27]. The authors introduced a self-organized division of labor framework, called partitioning social inhibition. Inspired by the division of labor in honeybees, each robot acquires its role according to a density-based distribution along a virtual segment. In this article, we also use a virtual space for coordination. However, there are several differences. First, we explore self-organization at the robot level rather than at the group level, which allows each robot to specialize inherently in different roles. Second, we focus on the generalization of the proposed communication module across different SR tasks. Third, Zahadat et al. [26],[27] restricted the virtual space to a segment, which is convenient for small swarms but may degrade performance as swarm size and role diversity increase. Here, we consider both 1-D and 2-D spaces, which significantly reduces the time required to switch between tasks and improves overall system convergence. Finally, in Section V, we experimentally demonstrate that our system achieves faster convergence in the self-organization process, a critical feature in SR systems. This article introduces a self-organized communication module for robot swarms that can successfully be used in diverse SR tasks. The main contributions are the following. 1) Transferability in SR: The communication is evolved only once and transferred to multiple SR tasks. 2) New Communication Paradigm: Introduction of a novel, self-organized, and decentralized communication module for robot swarms, based on a virtual construct that robots use to communicate, interact, and solve common tasks. 3) Performance and transferability is demonstrated in three popular SR tasks: a) aggregation; b) formation control; and c) foraging. 4) Scalability: The results show good performance and scalability in swarms of up to 60 robots, which lays the groundwork for solving many other SR tasks. II. COMMUNICATION MODULE Inspired by the fields of cooperative control [28] and graph neural networks [29], we propose a transferable decentralized communication module for solving SR tasks. It is based on a virtual communication state space, or simply communication space, where each robot has its own communication state. The communication space is a purely abstract construct with no direct physical counterpart—it does not represent a spatial mapping, topology, or replica of the real world in any way. The communication states are the only pieces of information communicated between the robots. Robots can change their communication state via virtual navigation. We categorize the communication module as the combination of the communication space, the robots’ virtual navigation in that space, and the communication of states between neighboring robots. A. Communication State Space Let Sbe the communication space. We consider the partition Rof Sintroduced as follows: R={R1,...,RM}|S=R1∪R2∪···∪RM.(1) Moreover, each region Rj, with j∈{1,...,M}, is represented by a centroid or virtual landmark lj∈Rj. Using these virtual landmarks, a convenient way to define the regions is shown as follows: Rj=s∈S|dSs,lj<dS(s,lk)∀k= j(2) where dS(s,s)refers to some distance metric within S.In the special case in which dS(s,lj)=dS(s,lk), the region is assigned randomly to one of the candidate regions. We define si∈Sas the communication state of the ith robot, considering i∈{1,...,N}, where Nis the number of robots or swarm size. Additionally, each robot has a virtual orientation (θi(t)), which defines its direction of movement within the virtual space S. A robot iis in region Rjat time instant tprovided that si(t)∈Rj. fR(si)=Rk⇐⇒ si∈Rk, where the function fRdetermines the virtual region to which a given robot’s state belongs. Each virtual region corresponds to a physical behavior. Moreover, robots communicate their individual state (coordinate in the communication space) to neighbors in range. Fig. 1(a) illustrates a 1-D communication space with six virtual regions {Ri}6 i=1represented by their associated virtual landmarks {li}6 i=1. In this example, there are eight robots whose communication states are represented by white circles. In this This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. SENDRA-ARRANZ et al.: EVOLUTION OF TRANSFERABLE AND SELF-ORGANIZED COMMUNICATION MODULES 3 Fig. 1. Hypothetical communication spaces and primitive selector. (a) Hypothetical 1-D communication space with six virtual regions (Ri)and their respective landmarks li(denoted as stars). (b) Hypothetical 2-D communication space with six virtual regions as in (a). In both (a) and (b), there are eight robots, whose communication states are represented as white circles. (c) Example of a primitive selector that maps virtual regions in Sto primitives fito be executed by the robots. (d) Screenshot of the eight robots executing their corresponding primitives. The geometric shape displayed on top of each robot represents the primitive being executed, while the color of the robot’s LED indicates its current virtual region (according to the colors shown in (a) and (b)). figure, there is one robot in R1,twoinR2,twoareinR3, two in R4, one in R6, and none in R5. Additionally, Fig. 1(b) illustrates the analogous 2-D communication space with the same number of virtual regions and landmarks. B. Connection Between Virtual Regions and Robot Behaviors Similar to the levels of competence in [30] and the behavior primitives in [31], we define a primitive pool F={f1,...,fK}, which is a set of primitive behaviors designed to solve specific and simple tasks. Some examples of primitives are “explore the arena,” “approach a light source” (phototaxis), “move away from a light source,” “follow another robot,” “stay in a nest,” and “forage from a resource area.” These primitives have access to the robot sensor readings and can modify the state of the robot’s actuators. In this article, we only consider manually designed primitives, but the set could also include evolved, learned, or hybrid primitives. Each virtual region is associated with exactly one behavior primitive. We therefore define a mapping between the virtual regions in Rand the primitives in F. This implies that a robot whose communication state belongs to a virtual region executes the primitive for that region. We refer to this mapping between the sets Rand F as the primitive selector. In this article, every robot has the same set of primitives, and the primitives and primitive selector are manually designed for each specific task being solved. An example of a primitive selector is illustrated in Fig. 1(c), where six virtual regions of either Fig. 1(a) or (b) are mapped onto four primitives {f1,...,f4}(represented graphically as different geometric shapes). Fig. 1(d) shows a swarm of eight robots that are solving a certain task. The primitive that each robot of the figure is executing is graphically represented with a 3-D geometry according to the robot’s communication state in Figs. 1(a) and (b) and the primitive selector in Fig. 1(c). C. Navigation in the Communication Space The communication state of each robot is subject to virtual navigation that modifies its coordinates in S. Furthermore, a navigation policy defines the rules and dynamics of virtual navigation, which are driven by a goal or convergence criterion. To be effective, such navigation policy should take into consideration the virtual landmarks and the communication states of the neighbors (with neighborhood in the communication space defined as follows) si,sjare neighbors ⇐⇒ fR(si)=fRsj.(3) In this article, we establish the navigation policy goal as follows: ∀si,sj∈{s1,...,sN},si= sj⇒fR(si)= fRsj.(4) This target condition is met when each robot’s communication state belongs to a unique virtual region, ensuring no two robots share the same region. To complement (4), notice that the virtual regions form a partition of S, so that any communication state always belongs to only one virtual region, regardless of the navigation policy. The navigation goal in (4) has been deliberatively established to solve the SR tasks of this article. However, alternative objectives can be designed to meet other task requirements (e.g., allowing multiple robots’ communication states to converge to the same virtual region). To fulfill the imposed policy, the virtual navigation is controlled by the communication controller (fcomm).The communication controller is a distinct unit and should not be confused with the conventional controller responsible for the robot’s physical behavior. The latter maps sensor readings to actuator actions, while the former governs the virtual navigation of the robots in the communication space. The communication controller of the ith robot is generically defined as follows: ai,θ tar,i=fcomm(si,scst,lcst,ltar)(5) where the outputs are the normalized navigation speed ai∈ [0,1] and the target virtual orientation θtar,iof robot iin the communication space S. Alternatively, the communication controller receives as inputs four vectors, namely, the closest This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 4IEEE TRANSACTIONS ON CYBERNETICS communication state among the neighbors in S(scst),the closest landmark (lcst, which is the landmark associated with the virtual region fR(si)), a target landmark (ltar), and its own communication state (si). The tuple (si,scst,lcst,ltar) summarizes all the information required by fcomm to fulfill (4), while avoiding the high dimensionality and dynamic size issues that would arise if all the virtual landmarks and all the neighbor communication states were provided. Before explaining the computation of the target landmark ltar,letpi∈NM 0(N0=N∪0)be a vector of priorities, where 0 represents the highest priority, and priority decreases with increasing numbers. The priorities in piare associated oneto-one with each of the virtual regions, so that pi(m)is the priority of region Rmfor all m∈{1,...M}according to robot i. The subscript iof piis used to denote that the priorities are subjective to each robot i∈{1,...,N}, so that each robot can have different virtual region priorities (see Sections II-D and II-E for more details). The target landmark (ltar)is selected according to the following criteria, listed according to their importance in the selection. 1) Whether the virtual region Rtar of ltar is already occupied by one or more neighbors. 2) The priority of the landmark’s region. 3) The distance, dS, from the robot’s communication state to the landmark. The communication controller is responsible for generating a virtual navigation velocity and orientation. The dynamics of the virtual orientation of a robot iare characterized as follows: τθ ∂θi(t) ∂t=θtar,i(t)−θi(t)(6) while the dynamics of the communication state are defined as follows: τs∂si(t) ∂t=ai(t)·cos(θi(t)) sin(θi(t)).(7) The fixed time constants τθand τsalso influence the linear and angular speeds of the virtual navigation, and are set to 20·∂tin both cases. Preliminary studies revealed that they are a good tradeoff between smoothness and fast response in the virtual navigation. Equations (6) and (7) are specifically designed for 2-D communication spaces. Thus, in the case of 1-D communication spaces, the dynamics reduce to (8) as there is no longer a need to include orientation τs∂si(t) ∂t=ai(t)·θtar,i(t). (8) In (8), the variable θtar,i∈{−1,1}has two possible values, each representing one of the two navigation directions along the single dimension of the communication space. Fig. 2(a) summarizes the stages of the virtual navigation process of a robot in the communication space. First, scst, lcst, and ltar are computed. Subsequently, these vectors are fed to the communication controller fcomm, which generates the desired navigation speed and virtual orientation. The communication state of the robot is ultimately updated based on (6) and (7). Fig. 2. Diagram of the overall system. (a) Communication module. (b) Robot controller. D. Robot Controller The robot controller, illustrated in Fig. 2(b), is responsible for the physical behavior of a robot. In each control cycle, the primitive selector is used to pick one of the primitives available inaprimitive pool, based on the current region of the robot in the communication space. Moreover, the selected primitive is executed using the current sensory readings, denoted as φ(t). In addition, the robot controller is also composed by the priority selector. A virtual region’s priority determines its relative importance for a given robot and is considered when the robot performs virtual navigation. The priority selector is defined as follows: pi(t)=PrioritySelector(φ(t))(9) so that its outcome is the vector pi, resulting from the concatenation of the priorities of all the primitives of the ith robot. The priorities can be set to fixed values, for instance, as in the aggregation and formation tasks (see Sections IV-A and IV-B). However, priorities can also be dynamically updated using conditional rules and the current sensor readings, φ(t).Asan example, a change in the priority of a primitive can happen when a robot runs low on battery and needs to critically prioritize a “recharge battery” behavior. In this scenario, the priority selector would simply check whether some battery level threshold was crossed, and modify the “recharge battery” priority accordingly. E. Interactions Between the Communication Module and the Robot Controller Fig. 2shows an overview of the communication module, the robot controller, and the data flow between them. The bidirectional interactions between these two components are the following. 1) Influence of the Communication Module on the Robot Controller: The communication module modifies the behavior of the robots in the physical environment through the primitive selector and the communication state. Specifically, the communication state of a This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. SENDRA-ARRANZ et al.: EVOLUTION OF TRANSFERABLE AND SELF-ORGANIZED COMMUNICATION MODULES 5 Fig. 3. Flowchart of the communication module phases. robot and the virtual region to which it belongs determine the physical behavior primitive executed by a robot. Similarly, transitioning from one virtual region to another also causes a change in the robot’s physical behavior. 2) Influence of the Robot Controller on the Communication Module: The robot controller can alter the virtual navigation that characterizes the communication module by means of the priority vector pi(t)local to each robot i. Consequently, pi(t)of a robot idetermines the virtual regions that this robot should occupy with highest priority at time instant t. The robot controller can modify the values of piat each control cycle based on the sensor readings. F. Evolution and Deployment Phases The parameters of the ANN defining the communication controller (fcomm)are optimized with the ultimate goal of satisfying (4). The communication controller is evolved prior to the design of the physical robot controller. Accordingly, it operates in two stages: 1) the evolution phase and 2) the deployment phase. The evolution phase focuses exclusively on optimizing the communication controller, where robots learn to navigate in the communication space. At this stage, the controller is agnostic to the SR task performed in the physical environment, and no primitive behaviors are associated with the virtual regions. The deployment phase begins once evolution is complete. Here, the communication module is configured for a specific SR task by assigning primitive behaviors to virtual regions and defining both the primitive selector and the priority selector. While this study employs manually designed primitives and selectors, these components could also be evolved or learned depending on task-specific requirements. A key advantage of the proposed module is its taskagnostic nature: it can be reused across diverse tasks without redesign, re-evolution, or retraining, a property we refer to as transferability [18]. Adapting the module to a new task requires only its configuration during deployment. In this study, we evolve a single communication controller and deploy it across multiple tasks: aggregation, formation control, and foraging (Fig. 3). Fig. 4. Example showing how the communication module and robot controller can solve a simple foraging. (a) Communication space showing example trajectories of two cooperating robots solving a simple foraging. (b) Primitive selector showing five primitive behaviors (fi)and how they are associated to the regions of the communication space. It also provides a short description of each goal of each primitive. (a) also shows multiple callouts with the values of the priorities of the robots during specific parts of the trajectories (illustrated with the style of the curves, being either solid, dashed, or dotted). For the sake of clarity, in this example we consider either ON and OFF priority values, so that ON means that the corresponding region has priority and the robot is attracted to it, and OFF implies that the corresponding region can be neglected. For example, the vector (OFF,OFF,OFF,OFF,ON) indicates that only region R5has priority, and, thus, the robot’s target behavior would be f5(“Return to nest”). G. Illustrative Example A simple multiplace foraging task is used as an example to illustrate the operation of the communication module. This foraging task is composed of a nest zone and two food areas. The aim of the robots is to find the food areas and, thereafter, transport as many food elements as possible to the nest by performing round trips between the corresponding resource area and the nest. Additionally, the robots must perform continual task allocation so that each robot seeks and forages from a different food area. This example with two robots is shown in Fig. 4.Fig.4(a) depicts the communication space and the virtual navigation of each of the two robots (as white and black curves, respectively). There are five virtual regions (R1,...,R5), each linked to a distinct primitive behavior (f1,...,f5)according to the primitive selector shown in Fig. 4(b). The callouts show the 5-D priority vectors with their values set to either OFF or ON for each of the five regions. The style of the curve (dotted, dashed, or solid) depicts the part of the trajectories in which the priorities of the callouts are used by each robot. The value OFF means that the robot’s virtual navigation will not be attracted toward it. For example, the vector (OFF,OFF,OFF,OFF,ON)indicates that only region R5has priority, while the rest can be neglected. Consequently, the This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 6IEEE TRANSACTIONS ON CYBERNETICS robot should navigate toward R5with the aim of executing behavior f5(“Return to nest”). Focusing on the trajectory of robot 1 (in white), the virtual navigation is the sequence {R1,R3,R5,R3,R5,...}, so that the primitives executed by this robot are {f1,f3,f5,f3,f5...}. In terms of robot behaviors, this sequence is as follows: 1) the robot searches for a food area by exploring the environment, and once food area 1 is found, 2) it continually transports resources from that food area to the nest. The virtual navigation of robot 2 (in black) is the series {R2,R4,R5,R4,R5,...}and the sequence of robot 2 primitives are {f2,f4,f5,f4,f5,...}. In this case, robot 2 starts executing the primitive “Find food area 2,” because robot 1 is already looking for the other food source. Subsequently, robot 2 performs a round trip foraging from the food area 2 and the nest. The values of the priority vectors play a crucial role in this example. For instance, the round trips between R3and R5are produced because when a robot acquires food, the robot controller updates the priority vector to (OFF,OFF,OFF,OFF,ON), indicating that only region R5 is active, and the new target behavior of the robot is returning to the nest (primitive f5). III. EVOLUTION OF THE COMMUNICATION CONTROLLER The communication module, fcomm, should enable robots to virtually navigate the communication space in a manner that satisfies (4).Forfcomm, we use an ANN whose parameters and topology are evolved using the neuroevolution of augmenting topologies (NEAT) algorithm [32].Even though fcomm can, in principle, be optimized using other evolutionary algorithms that target fixed neural structures, the use of NEAT is motivated by its ability to simultaneously evolve both ANN parameters and topology. NEAT begins with minimal architectures and progressively increases the number of neurons and synapses, seeking balance between performance and complexity. Moreover, NEAT incorporates niching strategies that promote model diversity and alleviate premature convergence to suboptimal solutions. In each generation of NEAT, every individual is evaluated in a robotics simulator (see Section IV). In each simulation, a swarm of ten robots navigates a virtual communication space divided into ten regions. At each control cycle, every robot transmits its communication state (si)to neighboring robots. One iteration of the communication module [with its control flow and main stages shown in Fig. 2(a)] is then computed, producing the next communication state for each robot. The objective of evolution in this study is to optimize the communication controller so that robots are distributed with exactly one robot per region, thereby satisfying (4). The initial communication states of the robots are uniformly distributed across the entire communication space. The coordinates of the virtual landmarks in the communication space are also randomly sampled, subject to a minimum distance constraint between landmarks. This minimum distance is a hyperparameter that depends on the total number of landmarks. For ten landmarks, we empirically set this value to 0.3, ensuring sufficient separation between landmarks. Fig. 5. Communication space configuration, topologies, and virtual navigation control. (a) Circle communication space embedded in a 3-D space. Remark: the thickness of the circle is only represented for visualization purposes, albeit the structure is a compact 1-D manifold. (b) Unfolded circle communication space. (c) Torus communication space embedded in a 3-D space. (d) Unfolded torus communication space. (e) Architecture of the CTRNN model used to control the virtual navigation of the robots in the communication space. A. Communication Space The communication framework is assessed in experiments with both 1-D and 2-D communication spaces. We define the 1-D communication space as the line segment S1= [−L/2,L/2)with the additional property that the two endpoints −L/2 and L/2 are connected—in fact, they are the same point in the communication space. Fig. 5(a) shows the 1-D communication space as a circle in a 2-D space, while Fig. 5(b) illustrates it as a line segment. Both representations correspond to the same communication space. The distance metric that characterizes S1is defined as follows: dS1s,s=mins−s,L−s−s(10) where s,s∈S1. The 2-D communication space is defined as an unfolded torus in S2=[−W/2,W/2)×[−H/2,H/2). The upper and lower boundaries of S2are connected, and the left and right boundaries are connected as well. Fig. 5(c) shows S2as a torus in a 3-D space, while Fig. 5(d) illustrates the unfolded torus. As in the 1-D case, the communication spaces represented in both figures are equivalent. The distance metric of S2is formulated as follows: dS2s,s=      ⎛ ⎝ mins1−s 1,W−s1−s 1 mins2−s 2,H−s2−s 2⎞ ⎠     2 (11) where s=(s1,s2)and s=(s 1,s 2)and s,s∈S2. B. Communication Controller The ANN model used in fcomm is the continuous-time recurrent neural network (CTRNN) [33], which is an ANN with feedback connections that operates in continuous time. The use of a CTRNN is motivated by the continuous dynamics of virtual navigation in the communication space, which presents a scenario where a reactive feed-forward model may not suffice. Fig. 5(e) shows the CTRNN architecture used in the experiments. The bottom nodes of the figure represent the input neurons of the neural network, and the top nodes represent the output neurons. Recall from Section II-C that fcomm receives This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. SENDRA-ARRANZ et al.: EVOLUTION OF TRANSFERABLE AND SELF-ORGANIZED COMMUNICATION MODULES 7 as input the closest neighbor’s communication state (scst),the closest landmark (lcst), and the target landmark (ltar).The CTRNNexecutionisprecededbyapreprocessingstageinwhich these vectors are converted into the distance and angle between the robot communication state and the corresponding vector. For instance, dS(si,scst)and ϕS(si,scst)are, respectively, the distance and angle between siand scst in S. In contrast, the outputs of the CTRNN are the virtual navigation velocity and target orientation for robot i. C. Evolutionary Setup The fitness function FFienforces that only one robot should occupy each virtual region in order to achieve the goal defined in (4). The instantaneous fitness function of a single robot i, used by the NEAT algorithm to optimize the CTRNN model, is shown as follows: FFi(t)=⎧ ⎨ ⎩ e−α·dS(si(t), l∗ i(t)),if sj/∈R∗ i(t)∀j= i 0,otherwise. (12) where R∗ i(t)is the virtual region to which the communication state of robot icurrently belongs (si∈R∗ iat time step t), and l∗ irefers to the virtual landmark corresponding to region R∗ i. The value of FFiincreases exponentially as the distance to l∗ idecreases, provided that robot iis the only robot whose communication state lies within virtual region R∗ i. The parameter αmodulates the rate at which fitness decays as the distance to the virtual landmark increases. Based on preliminary experiments, we set α=8toavoidsparse fitness landscapes while preventing robots from being overly rewarded when located at the borders of the virtual regions. The final fitness score of a simulation is computed as the average of FFi(t)across all robots in the swarm and all simulation time steps. Additionally, the fitness function is evaluated ten times (runs), with random and independent initializations. The NEAT population is composed of 100 individuals, with two elites preserved across generations. Maintaining two elites was found to improve performance in preliminary experiments. Each parameter of the CTRNN model is mutated with probability 0.1, using Gaussian mutation with a mutation power of 0.05. Furthermore, a new node is added with probability 0.05, and a new synaptic connection is created with probability 0.2. The remaining hyperparameters follow the configuration proposed by the authors of NEAT [32] and by previous studies of NEAT applied to SR [34]. Specifically, we use c1=1, c2= 1, and c3=0.4 as the compatibility coefficients, and δt= 3 as the compatibility threshold. These parameters are used during the speciation process to identify compatible species. The weights and biases are constrained to the range [−10,10] and initialized with a Gaussian distribution of zero mean and standard deviation σ=1. IV. EXPERIMENTS To assess the communication framework, we use three popular SR tasks: 1) aggregation; 2) geometric formations; and 3) foraging with robot battery dependence. We use the same communication module, evolved according to the details and hyperparameters presented in Section III, for all tasks, but configured with different behavior primitives, primitive selector, and priority selector for each task. All the experiments are implemented using the pybullet library [35] for realtime collision detection and multiphysics simulations. The robots are equipped with eight IR-based proximity sensors distributed along their perimeter, a GPS to know their absolute position in the arena, and communication transmitters and receivers that enable the sharing of their communication states with neighbors within range. In the case of the foraging experiment, each robot is also equipped with eight light sensors positioned along its perimeter and a ground sensor that detects the color of the floor below the robot. Communication ranges of 2 m are used in experiments with swarm sizes below 20, while the communication range is increased to 4 m when swarm sizes larger than 20 are considered. The robotics simulations, NEAT algorithm, CTRNN models, and robot and communication controllers are public and available in a github repository called Mereli.1 A. Aggregation in Groups In the aggregation task, robots have to form a desired number of groups with a given number of robots in each group. Moreover, the robots of each group have to aggregate as closely as possible to the other members of their group, while avoiding collisions. We define Gas the number of groups, and Gi,i∈{1,...,G}as the number of target members of each group. Thereafter, there are Gprimitives with the following meanings: fi−→ “Go to the center of mass coordinates of the neighbors also belonging to group i.” The number of virtual regions of the communication space equals the swarm size and, for each i, the primitive selector maps Gidifferent virtual regions to the primitive fi.Fig.6(a) illustrates an example of the aggregation in groups with 21 robots, G=3 groups, three primitives {f1,f2,f3}, and Gi= 7∀i. All the target groups of this task have the same importance and all the virtual region priorities are fixed to the same value of 1. Therefore, the priority selector used in this experiment is simply pi(t)=(1,...,1)∀t∀i∈{1,...,N}.(13) At the beginning of the task execution, the target number of groups and robots per group is manually set. The robots’ positions are initialized randomly within a square area of either 2×2 m for swarms with less than 20 robots, or 3 ×3m for swarm sizes greater than 20. The orientations of the robots are randomly initialized between 0 and 2π. The aggregation in groups is a highly important task that goes beyond simple aggregation, as it can be used to conveniently divide the labor of the swarm across teams in real-world geographically distributed tasks [36],[37], such as firefighting or search and rescue. 1https://github.com/Robolabo/Mereli/tree/VCommModule This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 8IEEE TRANSACTIONS ON CYBERNETICS Fig. 6. Illustration of the proposed SR experiments. (a) Aggregation in groups of desired sizes. (b) Formation of target swarm geometries. (c) Basic foraging with a nest area (black), two food areas (gray), and three red light sources where the robots can recharge their batteries. The aim is to perform a temporal role allocation so that there are always N/4 robots foraging from each of the resources (N/2 foragers in total), while the remaining swarm members are either recharging their batteries or waiting inside the nest. B. Formation The second task is the formation of desired swarm geometries (e.g., [38] and [39]). The formation is relative to the center of mass of the robots, implying that it can be established anywhere in the environment. The only requirement is that the distances between robots are approximately preserved according to the geometry of the specified formation. As an example, Fig. 6(b) illustrates a target formation with nine robots. Let the target formation of Npoints be formulated as follows: {x∗ 1(t),...,x∗ N(t)}={c(t)+1,...,c(t)+N}(14) where c∈R2is the center of mass of the swarm and ∀i∈ {1,...,N},i∈R2 +is the offset vector that defines the coordinates of the point x∗ irelative to cin the formation. In this experiment, the primitive selector is an injective mapping between R={R1,...,RN}and F={f1,...,fN}, so that each primitive has the following meaning: fi−→ “Go to coordinates x∗ i(t)=c(t)+i”. All the spots of the target formation have the same importance and, therefore, all the virtual region priorities are fixed to the same value of 1 [see (13)]. At the beginning of the task execution, the target formation geometry is manually set. Moreover, robots’ physical positions and orientations are randomly initialized as in the aggregation task (Section IV-A). C. Foraging The third task is foraging. The environment has a nest area and two resource areas located at opposites sides of the nest. Additionally, each robot is equipped with a battery that discharges when the robot is outside the nest and can be recharged in the proximity of a light source. The aim is to maintain 25% of the robots foraging from food 1 and another 25% that forage from food 2. The remaining 50% of the robots either wait inside the nest or recharge their batteries. Thus, there are four roles engaged by the robots, namely: stay in nest,forage food 1,forage food 2, and recharge battery.Note that this task requires dynamic role allocation because when a forager robot runs low on battery and goes to recharge, another idling robot has to update its role to forage in its place. Fig. 6(c) depicts the foraging experiment with N=12 robots accomplishing the task with the optimal distribution of roles. The figure also illustrates the locations of the nest (black ground area), the food areas (gray ground areas), and the red light sources where the robots can recharge their batteries. The battery level of each robot iis denoted as bi(t)∈[0,1] and is subject to the dynamics in (15) and (16) for its charge and discharge, respectively ∂b(t) ∂t=γ1−b(t)2(15) ∂b(t) ∂t=b(16) where γand bare the charge and discharge coefficients, and as initial conditions, we fix b(0)=1. There are N+1 virtual regions, and we define four behavior primitives f1−→ “Stay in the nest area,” f2−→ “Forage from food area 1,” f3−→ “Forage from food area 2,” f4−→ “Recharge battery (phototaxis”). Additionally, the primitive selector performs the following mapping between virtual regions and primitives: Rnest ={R1,...,RN 2}−→f1 Rfood1 ={RN 2+1,...,R3N 4}−→f2 Rfood2 ={R3N 4+1,...,RN}−→f3 Rbat =RN+1−→ f4. By default, the priorities of each virtual region are 2 for the regions in Rnest, 1 in the case of Rfood1 and Rfood2, and 3 for Rbat. Consequently, foraging from one of the food areas has the highest priority, staying in the nest has medium priority, and recharging the battery is a low-priority primitive. Nonetheless, when the condition bi(t)<0.4 is met, the priority of Rbat is updated to 0, turning battery recovery into the primitive with highest priority. When the battery level is above This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. SENDRA-ARRANZ et al.: EVOLUTION OF TRANSFERABLE AND SELF-ORGANIZED COMMUNICATION MODULES 9 Fig. 7. Results of communication module in isolation. (a) Results of three independent simulations of the virtual navigation in the 1-D communication space S1with ten virtual landmarks and ten robots. (b) Results of the virtual navigation in the 2-D communication space S2with 20 virtual regions and 20 robots. Red stars indicate the coordinates of the virtual landmarks and circles represent the communication states of the robots. (c) Box plot showing the evolution of the number of errors in the virtual navigation during a simulation with 30 robots. An error occurs when a robot moves into a virtual region that is occupied by another robot. 0.9 again, the priorities for the corresponding robot return to their default values. Specifically, the following priority selector is used in the foraging task: pi(t)= ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ ⎛ ⎜ ⎝1,...,1   N/2 ,2,...,2   N/2 ,3⎞ ⎟ ⎠,if battery recharged ⎛ ⎜ ⎝1,...,1   N/2 ,2,...,2   N/2 ,0⎞ ⎟ ⎠,if battery depleted. The environment is composed of a nest area (circular black ground area), two food areas (circular gray ground areas) of equal radius, and three red light sources that can be sensed from every point in the environment (see Fig. 6). The nest area is always located at the center of the arena and has a radius twice as large as the food areas. Specifically, the nest radius is fixed to 1 m for swarm sizes smaller than 40 and 2 m for swarms with more than 40 robots. The food areas are located symmetrically at either sides of the nest, being the nest-to-food distance: 3 m for swarms up to 40 robots, and 5 m for swarm sizes greater than 40. The robot positions are initialized randomly within a square centered at the nest’s origin, of either 2 ×2 and 3 ×3 for swarm sizes in the ranges [1,40)and [40,60], respectively. The battery is always fully charged at the beginning of every simulation, and γand b are always fixed to 0.01 and 0.001. V. RESULTS A. Communication Module in Isolation We first present the results of the evolved communication module in isolation, assessing whether the navigation goal in (4) is fulfilled during virtual navigation in both S1 and S2. Focusing first on the 1-D communication space S1, Fig. 7(a) shows the results of virtual navigation with ten virtual landmarks (red stars) and robots (white circles) across three independent executions (horizontal lines). In this case, the Fig. 8. Results of the aggregation in groups with swarms of 20 robots. (a) Aggregation in two groups of ten robots. (b) Aggregation in three groups with irregular sizes of 5, 6, and 9 robots. (c) Aggregation in four groups of five robots. (d) Aggregation in five groups of four robots. virtual navigation successfully meets the convergence criterion defined in (4). In addition, Fig. 7(b) depicts the results in S2using 20 virtual regions and 20 robots. Fig. 7(c) shows the evolution of the number of errors over simulation time steps, in a swarm of 30 robots using S2as the communication space. Every 100 time steps, the distribution of errors is summarized as a box plot based on 50 independent runs. The convergence time to reach a median error (red segments) of 5 is approximately 700 time steps, while achieving a median error of 0 requires around 2400 iterations. Moreover, the steady-state solution exhibits a stable median value of 0 and low variation. B. Aggregation in Groups Fig. 8shows the positions of the robots in the final frame of simulations with 20 robots and with different numbers of groups in the aggregation task. In all the runs, the task is successfully solved, demonstrating that the swarm is capable of allocating and distributing its members to different groups to fit the target sizes. Additionally, it should be noted that the clusters of aggregated robots tend to remain dispersed, preventing cluster mergers. C. Formation The swarm formation task is also solved with promising results. Fig. 9(a)–(e) illustrate the convergence toward four different formation geometries, showing the trajectories as well as the initial and final formations. In all representations, the colors of the circles indicate the time instant of the corresponding formation, according to the timeline in the color bar in Fig. 9(g). The short black lines denote the orientation of the robots. All formations are reached with high accuracy, starting from randomly selected initial positions. Moreover, Fig. 9(f) presents a simulation in which the target formation is switched three times during the experiment. The desired formation geometry is set by explicitly updating the i vectors of the primitives (see Section IV-B). In real-world applications, the target geometry update could be teleoperated or preprogrammed. In addition to formation switching, Fig. 9(f) also shows coordinated movement of the swarm while preserving the converged formation. Recall from Section IV-B that the aim of the primitives is to drive the robots toward the formation points {c(t)+1,...,c(t)+N}. Thus, once the swarm has successfully converged to the desired formation, the entire swarm can be controlled by simply adding an offset or bias This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.