Collaborative learning model predictive control for repetitive tasks

Collaborative learning model predictive control for repetitive tasks Chanfreut, P., Maestre, J. M., Camacho, E. F., and Borrelli, F. Abstract— This paper presents a cloud-based learning model predictive controller that integrates three interacting components: a set of agents, which must learn to perform a finite set of tasks with the minimum possible local cost; a coordinator, which assigns the tasks to the agents; and the cloud, which stores data to facilitate the agents’ learning. The tasks consist in traveling repeatedly between a set of target states while satisfying input and state constraints. In turn, the state constraints may change in time for each of the possible tasks. To deal with it, different modes of operation, which establish different restrictions, are defined. The agents’ inputs are found by solving local model predictive control (MPC) problems where the terminal set and cost are defined from previous trajectories. The data collected by each agent is uploaded to the cloud and made accessible to all their peers. Likewise, similarity between tasks is exploited to accelerate the learning process. The applicability of the proposed approach is illustrated by simulation results. I. INTRODUCTION In the last decades, model predictive control (MPC) has gained increasing acceptance in both industrial and academic fields, and is now established as a major methodology for dealing with multivariate and constrained systems [1], [2]. MPC policies are based on reiterative computations of the sequence of inputs that optimizes the system performance during a future time horizon, thus providing a unique anticipation capacity. The increased capability of sensing, computing, and storing data, together with the powerful advances in machine learning techniques, have boosted the application of datadriven methods within the field of MPC [3]. In this paper, we use the learning model predictive control (LMPC) formulation presented in [4]. This strategy focuses on systems with a strong repetitive behaviour, such as autonomous racing cars [5], and counteracts the inherent finite-horizon nature of MPC controllers. The learning process is based on the systematic design of the terminal set and terminal cost function of the MPC problem by using data. Note that the idea of learning the objective function is also present in other control approaches such as inverse optimal control [6]. Under the assumption that the model is perfectly known, LMPC controllers [4] are proved to progressively improve the cost of P. Chanfreut, J. M. Maestre and E. F. Camacho are with the Department of Systems and Automation Engineering, University of Seville, Spain (emails: {pchanfreut,pepemaestre,efcamacho}@us.es) F. Borrelli is with the Department of Mechanical Engineering, University of California at Berkeley, Berkeley, CA 94701 USA (e-mail: [email protected]) This work is supported by the Spanish Training Program for Academic Staff under Grant FPU17/02653, by the Manuel Gay´ an Buiza Award, by the European Research Council Advanced Grant OCONTSOLAR under Grant SI-1838/24/2018, and by the Spanish MCIN/AEI/10.13039/501100011033 Project C3PO-R2D2 under Grant PID2020-119476RB-I00. Also, we would like to thank Dr. Filiberto Fele for his feedback regarding the article. executing a given task, and closely approximate the solution providing the optimal performance. Recent efforts have been made to extend the results in [4] to decentralized systems and to a more flexible paradigm where the repeated tasks are more varied. For example, [7] presents a task decomposition method where they are defined as aggregations of subtasks in different orders, and [8] considers the case of periodically time-varying systems. See also [9], where a decentralized LMPC for nonlinear multi-agent systems with coupled state constraints is introduced. This article presents a cloud-based LMPC for multi-agent systems that repeatedly execute a finite set of tasks, which consist in driving admissibly the agents’ state to a certain target point. The tasks conditions can change with time and thus modify the state restrictions to accomplish them, for instance, consider a mobile robot driving iteratively a road segment with and without the presence of obstacles. To deal with static obstacles, we define different modes of operation, which accordingly impose different constraints in the MPC problems. In addition, a reactive strategy is included to avoid collisions with moving obstacles, which can model for example humans walking. The main novelty of the paper is the use of a data cloud in multi-agent LMPC- based systems as a means to enable collaboration while keeping the computation of the inputs decentralized. In this regard, all agents upload their collected data to a common cloud, so that it can also be used by their peers. Moreover, the data collected when executing a certain task are analyzed to check if they can also be exploited for learning similar ones. The proposed approach guarantees recursive feasibility, asymptotic stability of the target states, and a non-increasing evolution of the cost of executing the same task without moving obstacles. In this respect, under mild assumptions, moving obstacles are proven not to jeopardize persistent feasibility or convergence to the targets with the proposed controller. The rest of the paper is organized as follows. Section II describes the system dynamics and the control goal. Section III introduces the proposed collaborative learning approach for iterative tasks. Section IV presents its theoretical guarantees. Section V provides the simulation results, and, finally, Section VI presents our conclusions and future work prospects. II. PROBLEM FORMULATION Consider a set N={1,2, . . . , Nag}of identical agents with linear time-invariant dynamics. Without loss of generality, they will represent mobile agents navigating in a two-dimensional space. In particular, the continuous-time arXiv:2211.16633v1 [eess.SY] 29 Nov 2022 dynamics of all i∈ N are given by:1     ˙sx,i ˙sy,i ˙vx,i ˙vy,i     | {z } ˙xi =    0010 0001 0000 0000        sx,i sy,i vx,i vy,i     | {z } xi +    0 0 0 0 1 0 0 1    ax,i ay,i |{z} ui ,(1) where sx,i,vx,i, and ax,i are respectively the position, velocity and acceleration of agent iin the horizontal direction, and sy,i,vy,i, and ay,i in the vertical direction. Hereafter, consider a discrete-time version of (1) and let us use time index k, hence, xi(k)and ui(k)denote respectively the state and input of agent iat time instant k. Also, define si(k) = [sx,i(k), sy,i(k)]Tand vi(k) = [vx,i(k), vy,i(k)]T, and consider set ˆ X={ˆx1,...,ˆxp,...,ˆxq,...,ˆxNt}, which contains a finite number Ntof target states. Finally, assume that the initial state of every agent i∈ N is such that xi(0) ∈ ˆ X, and xi(0) 6=xj(0), for all j∈ N \ {i}. A. Tasks and constraints The goal of the agents is to travel iteratively between the points in ˆ Xwhile optimizing a given performance criterion and satisfying state and input constraints. Definition 1. Task p→qdenotes the intent of traveling from ˆxpto target state ˆxqin an admissible way, with ˆxp,ˆxq∈ˆ X. In this regard, all agents i∈ N must satisfy vi(k)∈ V =[vx,i, vy,i]T| vx,i, vy,i 2≤vmax,(2a) ui(k)∈ U =[ax,i, ay,i]T| ax,i, ay,i 2≤amax,(2b) for all k≥0, where vmax and amax are the maximum velocity and acceleration. On the other hand, the constraints on their position depend on the task that is performed. In particular, to perform p→q, the agents should travel along a path represented by set Rp→q(see Fig. 1). Moreover, certain zones of Rp→qmay become non-transitable for a number of time steps, e.g., due to the presence of static obstacles. The latter shrinks the set of admissible positions, thus modifying the state constraints. It is assumed that these changes can be classified into a finite number of modes of operation for each task. In this regard, consider the following: Definition 2. The set of admissible states of any agent i when performing task p→qin mode mis defined as: Xm p→q={xi|si∈ Rp→q\ Om p→q, vi∈ V}, where Om p→q⊂ Rp→qis the set of forbidden positions. In addition, the agents may encounter moving obstacles while executing their tasks, e.g., pedestrians crossing if they represent self-driving cars. Finally, let us introduce the following assumptions: Assumption 1. The mode mis known at time 0 of each task by the agents and does not change during its execution, 1The approach proposed in this article may be similarly applied to other linear time-invariant systems. However, for the sake of convenience and clarity, we will consider dynamics (1) throughout the entire manuscript. whereas the moving obstacles are discovered while the agents perform the tasks. Assumption 2. The agents know an initial feasible state trajectory χm p→qand its corresponding input sequence υm p→q to complete any task p→qin any of its possible modes m without considering moving obstacles. Assumption 3. The moving obstacles move along an assigned path, e.g., a crosswalk in the case of pedestrians. B. Control goal The control goal is to design the agents’ MPC problems such that they perform optimally any task p→qin any of its modes of operation m, and can avoid collisions with moving obstacles. In this regard, the stage performance cost for any task with target ˆxqis assumed to be defined as h(xi(k), ui(k),ˆxq) = kxi(k)−ˆxqk2 Q+kui(k)k2 R,(3) where Qand Rare positive definite matrices. Therefore, h(·) satisfies h(ˆxq,0,ˆxq)=0and h(xi(k), ui(k),ˆxq)>0for any xi(k)6= ˆxq,ui(k)6= 0. III. COLLABORATIVE TRAJECTORY OPTIMIZATION BASED ON LMPC In this paper, the inputs optimization is based on the LMPC presented in [4]. This controller learns its optimal terminal constraints and cost to admissibly perform a task. The formulation is characterized by the following: (i) The terminal set is a sample safe set that is built up from previous successful trajectories. (ii) The terminal cost weights the cost-to-go from the terminal state to the desired target according to the data collected in previous repetitions of the tasks. Let us define the sequences of visited states and implemented inputs in the j-th task performed by agent ias xj i=hxi(tj i), . . . , xi(Tj i)i,uj i=hui(tj i), . . . , ui(Tj i)i,(4) where tj iand Tj irepresent respectively the instants in which i started and finished its task number j. Likewise, let mj ibe the mode in which iexecuted its j-th task, and consider set Mm i,p→q=nj|xi(tj i)= ˆxp, xi(Tj i)= ˆxq, mj i=mo,(5) which contains the indexes of the tasks in which agent i performed p→qin mode m. Fig. 1. Sketch of the system. The tasks consist in traveling admissibly between the target points (green marks). The forbidden areas (dark grey zones) depend on the mode mof each task and modify the state constraints. A. Cloud storage As shown in Fig. 1, the proposed approach integrates the set of local agents, a coordinator, and a cloud database. The cloud stores the data that allows the agents to build up the terminal safe set and terminal cost as described in [4]. In this regard, consider the following assumption: Assumption 4. All agents i∈ N collect and upload to the cloud their sequences of visited states and incurred costs when performing their tasks. Regarding the terminal set, any agent ican derive a sample safe set SSm i,p→qfor task p→qin mode mby collecting all its visited states when executing it, i.e.2, SSm i,p→q=  [ j∈Mm i,p→q Tj i [ t=tj i xi(t)   .(6) Note that for every xi(t)∈ SSm i,p→q, with t∈[tj i, T j i], there exists a sequence of inputs that drives the subsystem state to target ˆxqsatisfying the constraints of mode m, i.e., [ui(t), . . . , ui(Tj i)]. Accordingly, all states in sequences xj i, for all j∈ Mm i,p→q, belong to the maximal control invariant set associated with task p→qand mode m. Also, to define the terminal cost function, we store for all xi(t)∈ SSm i,p→qthe corresponding cost-to-go to complete the task, i.e., Fi(t) = PTj i k=th(xi(k), ui(k),ˆxq), where t∈[tj i, T j i]and j∈ Mm i,p→q. Remark 1. Since all the agents are dynamically identical, set SSm i,p→qalso contains safe states for any other agent j∈ N \ {i}. Therefore, one can define a common sample safe set for all agents as: SSm p→q=([ i∈N SSm i,p→q).(7) For the same reasons, the costs-to-go Fi(·)also provide useful information for all j∈ N \ {i}. Remark 2. By exploiting the similarity between tasks, it is possible to use data collected in a certain task to learn a different one. For example, consider the system in Fig. 1 and assume that the j-th task performed by agent iwas p→q. Additionally, consider a different task r→n, and assume that roads Rp→qand Rr→nare parallel and satisfy Rr→n={(sx, sy) + λ: (sx, sy)∈ Rp→q}, where λis the shift vector. Then, trajectories uj iand xj i+λ= [xi(tj i) + λ, xi(tj i+ 1) + λ, . . . , xi(Tj i) + λ] provide a new candidate solution for task r→n. One should simply check if it satisfies the constraints of some of the modes of r→n, and add the new solution to the corresponding safe set if admissible. Note also that if the problems associated with tasks p→qand r→nare convex, 2For the sake of simplicity, we have omitted the time index in SSm i,p→q, but note that it represents the safe set according to the trajectories realized up to current instant k. then a convex combination of the trajectories obtained for p→qshifted by λalso provides safe points for task r→n. B. Local controllers Consider any agent i∈ N, let p→qbe its assigned task at time instant k, and mbe the currently active mode for this task. Then, the input implemented by agent iis computed by solving the following problem: min ui k+N−1 X t=k h(xi(t|k), ui(t|k),ˆxq) + Pm p→q(xi(k+N|k)) s.t. xi(k|k) = xi(k),(8a) xi(t+ 1|k) = Ax(t|k) + Bui(t|k),(8b) xi(t+ 1|k)∈ Xm p→q, ui(t|k)∈ U,(8c) ∀t∈[k, ..., k +N−1],(8d) xi(k+N|k)∈ SSm p→q,(8e) where Nis the prediction horizon, uiis the sequence of inputs from instant kto k+N−1, and (t|k)denotes a prediction for time instant tmade at k. Additionally, function Pm p→q(·)assigns to every point in SSm p→qthe minimum cost to complete the task according to the data collected up to instant k. Note that this value can be computed from the costs realized in previous executions of the tasks. 1) Moving obstacles: As mentioned in Section II-A, in addition to the modes, the agents may encounter moving obstacles while executing the tasks. In this regard, consider that agent iis performing task p→qin mode mby solving (8), and that it detects a moving obstacle at instant k. Then, it switches to the following modified problem: min ui k+N−1 X t=k h(xi(t|k), ui(t|k),ˆxq) s.t. xi(k|k) = xi(k),(9a) xi(t+ 1|k) = Axi(t|k) + Bui(t|k),(9b) xi(t+ 1|k)∈ Xbo,m p→q, ui(t|k)∈ U,(9c) si(k+N|k)∈Proys(SSm p→q),(9d) ∀t∈[k, ..., k +N−1],(9e) where Proys(·)denotes the projection onto the components associated with the position. Also, Xbo,m p→qrepresents the subset of Xm p→qsuch that Proys(Xbo,m p→q)is the area of the road before the path along which the obstacle moves (see Assumption 3). Note that the speed is not constrained in (9d) since agent imay need to slow down to remain in this area. Once the moving obstacle is no longer detected, it starts solving again (8) to continue traveling to target ˆxq. C. Algorithm Algorithm 1 provides the pseudo-code of the procedure followed by each agent i∈ N in the case without moving obstacles. This procedure can be run in parallel by all agents in the system. Note that the sharing of information is involved in the download/upload of data from/to the cloud, which only occurs at the beginning and end of the tasks. That is, there is no agent-to-agent communication, and they may also access the cloud at different time instants. In addition, note that the goal of the coordinator is only to assign the tasks to the agents and inform them of the mode of operation.3 Algorithm 1 Initialize SSm p→qfor all tasks and modes using initial solution χm p→q, and define Nfree as the set of agents that are ready to be assigned a task. Also, consider some agent i∈ Nfree, let its state be ˆxp∈ X , and jits tasks’ counter. Then, at each step k, consider the following: 1: if i∈ Nfree then 2: The coordinator assigns a new target state ˆxq∈ˆ X to i, with q6=p, and informs it about the mode m. 3: Agent iaccesses the cloud to get safe set SSm p→q, and saves the starting instant of the task, i.e., tj i=k. 4: The coordinator updates Nfree =Nfree \ {i}. 5: end if 6: Agent ifinds its input by solving (8) for its assigned task, and updates its state. 7: if the task is completed then 8: Agent iuploads to the cloud its realized states xi(t), for t∈[tj i, k], and they are added to SSm p→q. 9: The costs-to-go Fi(t), for t∈[tj i, k], are also computed and uploaded to the cloud. 10: It is checked whether the new data can be exploited for other tasks. 11: Set j=j+ 1,Nfree =Nfree ∪ {i}, and p=q. 12: end if Remark 3. Detecting a moving obstacle only implies a change in Steps 6, 8 and 9 of Algorithm 1. In particular, instead of solving problem (8), the agents would use (9) until it stops being detected. Likewise, the data altered by the presence of moving obstacles should not be uploaded to the cloud for its use to define the terminal cost and set of (8). Finally, note that there is no change of constraints during the prediction horizon, that is, the agents either solve (8) or (9). IV. THEORETICAL PROPERTIES This section describes the theoretical properties of the proposed controller. Let [x∗ i(k|k), x∗ i(k+ 1|k), ..., x∗ i(k+ N|k)] be the optimal state sequence computed by agent iat time instant k, and [s∗ i(k|k), s∗ i(k+ 1|k), ..., s∗ i(k+N|k)] the associated sequence of positions. Similarly, for state sequence [xi(k), xi(k+ 1), ..., xi(k+t)] with t > 1, vector [si(k), si(k+ 1), ..., si(k+t)] is the corresponding position sequence. Finally, let us introduce the following assumptions: Assumption 5. Any agent i∈ N is able to detect moving obstacles before the Euclidean distance between iand the obstacle is lower than or equal to ∆, with ∆being the maximum distance the agents can travel in Nsteps. 3The analysis and optimization of the tasks’ assignments have been left out of the scope of this paper. Assumption 6. Consider a sequence [xi(k), xi(k+ 1), . . . , xi(k+t)] such that xi(k+n)∈ X bo,m p→q⊂ Xm p→q for all n= 0, ..., t. Then, it is possible to travel from position si(k)to si(k+t)in T≥ttime steps following a state sequence that also remains in Xbo,m p→q. Assumption 7. Let xi(k)be such that si(k)∈ Proys(SSm p→q). Then, there exists an admissible sequence of inputs such that xi(k+t)∈ SSm p→qfor some t≥0. Then, the following theorems hold. Theorem 1 (Recursive feasibility).All agents i∈ N can find a feasible solution of their optimization problems at all instants k≥0. Proof. This proof is divided in two parts. First, we focus on the case in which there are no moving obstacles. Consider that at time instant kagent i∈ N is assigned task p→qin mode m. Also, assume that the latter was also the z−th task previously completed by agent l. Then, the first Nstates of trajectory xz land its associated input sequence (see (4)) provide a feasible solution of problem (8) for agent iat k. Also, if no agent has performed task p→q in mode mbefore, the similar result can be derived by using the initial solution (χm p→q,υm p→q)(see Assumption 2). Now let us move to instant k+1. Given (8e), x∗ i(k+N|k) belongs in an admissible state trajectory to complete the task. Let x∗ i(k+N|k)correspond to the state at instant tof some agent l, and take the following candidate solution: x∗ i(k+ 1|k)x∗ i(k+ 2|k). . . x∗ i(k+N|k)xl(t+ 1), u∗ i(k+ 1|k)u∗ i(k+ 2|k). . . u∗ i(k+N−1|k)ul(t).(10) Given that xi(k+1) = x∗ i(k+1|k), that sequences computed at kare admissible, and that xl(t+ 1) ∈ SSm p→q, we have that (10) provides a feasible solution of problem (8) at time instant k+1. By induction, recursive feasibility is guaranteed. The recursive feasibility proof when moving obstacles come into play is based on Assumptions 5, 6 and 7. Consider that agent isolves (8) at instant k+τ, with τ > 0. If at k+τ+ 1 it detects a moving obstacle, it switches to problem (9). From the solution of (8) at k+τ, and considering Assumption 5, we have xi(k+τ+ 1) = x∗ i(k+τ+ 1|k+τ),(11a) x∗ i(k+τ+t|k+τ)∈ Xbo,m p→q,∀t= 1, ..., N, (11b) x∗ i(k+τ+N|k+τ)∈ SSm p→q.(11c) Given (11a), (11b), and Assumption 6, it is possible to find a sequence of inputs to go from si(k+τ+ 1) to s∗ i(k+τ+N|k+τ)in Nsteps while remaining in the area of the road before the path along which the obstacle moves. Also, from (11c), we have that s∗ i(k+τ+N|k+τ)∈ Proys(SSm p→q), hence the latter would provide a feasible solution of problem (9). Finally, assume that the moving obstacle stops being detected at k+ Γ + 1, with Γ> τ, and, therefore, agent iswitches back to problem (8). The only constraint that may compromise feasibility is (8e). Under Assumption 7, agent ican find an admissible trajectory to go from terminal state x∗ i(k+ Γ + N|k+ Γ) to the safe set, e.g., by traveling to positions in Proys(SSm p→q)while progressively adjusting its speed. Likewise, the solution at k+ Γ provides an admissible input sequence to get from xi(k+ Γ + 1) to x∗ i(k+ Γ + N|k+ Γ). By aggregating both input sequences, a feasible solution of problem (8) at k+ Γ + 1 is defined for a certain horizon. Note that one could recalculate (8) using a larger Nif it is necessary.  Theorem 2 (Stability for each task).For any task p→qand mode m, the equilibrium point ˆxqis asymptotically stable. In the no moving obstacles case, asymptotic convergence to the target is proven by showing that the optimal cost is a Lyapunov function for equilibrium point ˆxq, with the cost function being the objective in (8). The latter is proven by following the same steps as in [4, Thm. 1]. On the other hand, if agent idetects a moving obstacle, the value of the cost function in (8) may increase while its inputs are computed by solving (9). Nonetheless, assuming that the detection of the latter is sufficiently spaced out in time, asymptotic stability of the target points can also be proven. In particular, consider that the last moving obstacle stops being detected at instant k+Γ+1, then, from that instant until the end of the task, agent iwill always use (8). During this period, the decreasing evolution of the cost function will be guaranteed, and we can use the same reasoning as in [4, Thm. 1] to conclude that iwill converge to target ˆxq. Theorem 3 (Non-increasing costs).The costs of performing any task p→qin mode mwithout moving obstacles decrease with the number of repetitions, regardless of the agents that perform it. The proof of Theorem 3 can be easily derived from [4, Thm. 2]. In particular, using [4, Thm. 2], it is straightforward to prove that the cost of executing a given task decreases as some agent repeats it. Since in the proposed scheme all agents are identical and share their collected data, this nonincreasing property holds equally. Considering the theorems above and assuming that the agents converge to a steady-state trajectory for each p→q and mode m, it is possible to find a link with the associated infinite-horizon problem [4, Thm. 3]. In particular, the steady-state trajectory is proven to match the optimal solution of a finite-time approximation of the latter if it is convex. V. SIMULATION RESULTS Consider a system where 3 agents travel between a set of predefined locations by using the cloud-based LMPC described above. Also, assume that the plant layout is as shown in Fig. 2, where the intersections correspond to the target locations. The conditions for performing the tasks vary between two modes of operation, i.e., m={0,1}. These modes change independently for each of the tasks since the conditions in each road can change at different time steps. In addition, we consider that at the middle of each road there is a crosswalk where the agents find pedestrians, which are handled as moving obstacles, with a probability of 10%. 123457698 1 2 3 4 5 6 7 8 9 22 21 21 27 19 27 23 19 19 21 16 15 17 18 17 15 16 17 17 15 17 14 15 17 15 20 25 0 Fig. 2. Layout of the plant. The dark grey areas indicate the forbidden zones, which vary in time and determine the modes of operation. Fig. 3. Agents’ position trajectories in each repetition of different tasks. As the agents get and share more data, their trajectories are progressively shifted from the initial feasible solution towards the optimal one. The parameters of the simulation are: vmax = 3m/s, amax = 1.5m/s2,N= 4,Q= 0.01I4, and R= 0.5I2, where I4and I2represent respectively the identity matrices of dimensions 4×4and 2×2. The discrete model of the agents was obtained by discretizing (1) with a sample time of 1.5s. Also, the condition to terminate any task with target ˆxq is defined as ksi−ˆsqk∞≤0.01 and kvi−vqk∞≤0.001 for all i. Finally, the safe sets were computed as in (7), and we exploited the parallelism between tasks (see Remark 2). The results obtained are summarized below45. Fig. 3 illustrates the evolution of the agents’ position for different tasks. The significant difference between consecutive repetitions that can particularly be seen in the case of 2→3and 6→9reflects the effect of using data from parallel tasks to accelerate the learning process. That is, between two executions of the same task, the agents may perform others providing useful data for the one at issue. Notice that given footnote 5, the constraints consider a circular area surrounding the dark grey zones as forbidden. Also, note that for each p→qand mode m, the optimal trajectory is considered to be the one obtained by optimizing the sum of stage cost (3) for a very long prediction horizon subject to the corresponding constraints. Fig. 4 compares the cost of executing two different tasks with the results obtained using the original LMPC in [4], 4To reduce the computational complexity, we have used the convex approximation of the safe sets when solving (8) (see [10]). 5To implement the state constraints in (8c) we have imposed kxi(t|k)− cm p→qk ≥ Dm p→q, for all t=k+1, ..., k+N, where cm p→qis the Chebyshev center of the forbidden area and Dm p→qa properly defined distance. Fig. 4. Evolution of the costs of performing different tasks with the proposed cloud-based approach and when using the original LMPC formulation [4]. The number of repetitions of each task varies because the coordinator assigned them randomly. i.e., the same formulation but without considering the storage cloud and the exploitation of the similarity between tasks. For the sake of convenience, we use CB-LMPC to refer to the proposed cloud-based approach, and simply LMPC to refer to [4]. As can be seen, the costs follow a decreasing trend and tend to stabilize at the same value for a given p,q, and m. Nevertheless, the CB-LMPC allows us to obtain lower costs during the learning process, and to converge to a solution in a reduced number of repetitions. In addition, Fig. 5 illustrates the loss of optimality incurred by the agents in their tasks with the CB-LMPC. Note that there are 48 possible tasks (12 roads, two ways, and two modes), and that each agent performed about 130. Therefore, on average, they repeated less than 3 times each possible task. It should also be remarked that Fig. 4 and 5 focus on tasks not affected by pedestrians, since they are the ones that should lead to non-increasing costs. Finally, Fig. 6 (left) illustrates the effect of detecting pedestrians on the agents’ position. As can be seen, it remains approximately constant between instants 10s and 18s, which corresponds to the waiting period while the pedestrian crosses. Also, Fig. 6 (right) illustrates the excess of costs over the optimal one in tasks affected by pedestrians more than once. It can be seen how the data collected to optimize performance without pedestrians helps indirectly to reduce the costs in this case. VI. CONCLUSIONS In this paper, a LMPC for multi-agent systems that perform repetitive tasks is presented. The agents collect their state trajectories and incurred costs, and upload them to a common cloud to help their own and their peers’ performance. Moreover, we have distinguished different modes of execution of these tasks, which modify the state constraints to deal with static obstacles. Also, we have included a strategy for reacting to moving obstacles, which is guaranteed not to compromise recursive feasibility under the introduced assumptions. The data in the cloud are in turn analyzed to check if some trajectories can be concurrently exploited for more than one task. Our results show that the presented LMPC can notably improve the learning rate in a context where a significant number of tasks are alike. Fig. 5. Losses of optimality obtained by agents 1 (left), 2 (middle), and 3 (right) in their tasks. The lines show the corresponding moving averages considering a sliding window of 5 tasks. Fig. 6. Deviation of the position evolution (left) and extra costs (right) caused by the presence of pedestrians. The left plot shows the position of agent 2 when it encounters a pedestrian while performing task 2→1, and the result at convergence for the case without pedestrians. Future work will extend the proposed architecture to systems where the agents’ dynamics and constraints are not identical. Additionally, we will integrate the possible presence of moving obstacles directly in the LMPC formulation to guarantee non-increasing costs also in this case. REFERENCES [1] E. F. Camacho and C. Bordons, Model Predictive Control,2nd ed., ser. Advances Textbooks in Control and Signal Processing. London, UK: Springer, 2013. [2] S. J. Qin and T. A. Badgwell, “A survey of industrial model predictive control technology,” Control engineering practice, vol. 11, no. 7, pp. 733–764, 2003. [3] L. Hewing, K. P. Wabersich, M. Menner, and M. N. Zeilinger, “Learning-based model predictive control: Toward safe learning in control,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 3, pp. 269–296, 2020. [4] U. Rosolia and F. Borrelli, “Learning model predictive control for iterative tasks. a data-driven control framework,” IEEE Transactions on Automatic Control, vol. 63, no. 7, pp. 1883–1896, 2017. [5] U. Rosolia, A. Carvalho, and F. Borrelli, “Autonomous racing using learning model predictive control,” in 2017 American Control Conference (ACC). IEEE, 2017, pp. 5115–5120. [6] N. Ab Azar, A. Shahmansoorian, and M. Davoudi, “From inverse optimal control to inverse reinforcement learning: A historical review,” Annual Reviews in Control, vol. 50, pp. 119–138, 2020. [7] C. Vallon and F. Borrelli, “Task decomposition for iterative learning model predictive control,” in 2020 American Control Conference (ACC). IEEE, 2020, pp. 2024–2029. [8] N. Scianca, U. Rosolia, and F. Borrelli, “Learning model predictive control for periodic repetitive tasks,” in 2020 European Control Conference (ECC). IEEE, 2020, pp. 29–34. [9] E. L. Zhu, Y. R. St¨ urz, U. Rosolia, and F. Borrelli, “Trajectory optimization for nonlinear multi-agent systems using decentralized learning model predictive control,” in 2020 59th IEEE Conference on Decision and Control (CDC). IEEE, 2020, pp. 6198–6203. [10] U. Rosolia and F. Borrelli, “Learning model predictive control for iterative tasks: A computationally efficient approach for linear system,” IFAC-PapersOnLine, vol. 50, no. 1, pp. 3142–3147, 2017.