Robust learning-based MPC for nonlinear constrained systems

Robust learning-based MPC for nonlinear constrained systems Jos´ e Mar´ ıa Manzano a, Daniel Limon a, David Mu˜ noz de la Pe˜ na a, Jan-Peter Calliess b aDepartamento de Ingenier´ ıa de Sistemas y Autom´ atica. Universidad de Sevilla. Camino de los Descubrimientos s/n 41092 Sevilla, Spain bOxford-Man Institute of Quantitative Finance. University of Oxford, Eagle House, Walton Well Road, Oxford, OX2 6ED, UK Abstract This paper presents a robust learning-based predictive control strategy for nonlinear systems subject to both input and output constraints, under the assumption that the model function is not known a priori and only input-output data are available. The proposed controller is obtained using a nonparametric machine learning technique to estimate a prediction model. Based on this prediction model, a novel stabilizing robust predictive controller without terminal constraint is proposed. The design procedure is purely based on data and avoids the estimation of any robust invariant set, which is in general a hard task. The resulting controller has been validated in a simulated case study. Key words: Predictive control; Learning control; Robust stability; Nonlinear systems; Lyapunov stability 1 Introduction Model-based control design, and particularly model predictive control (MPC), relies on the availability of an accurate description of the plant. When a model of the plant dynamics is unavailable a priori, system identification methods can be employed to devise such models automatically from observational data. The objective of this paper is to design a predictive controller based on such a learning method. In this setting, the learning method should be flexible enough to learn rich classes of dynamical systems, while at the same time, it should offer bounds on its predictive performance. The latter is important in the predictive control setting if one wishes to give guarantees on the performance and feasibility of the data-based controller, e.g. for nonlinear [1] or ciber-physical systems [2]. Learning and data-driven predictive controllers have recently gained the attention of the control community [3]. An approach to learning-based MPC that is independent of the concrete learning paradigm was proposed in [4]. A broad scope opens up when considering the learning methodology: some research consider direct weight optimization methods [5, 6], others Gaussian processes [7, 8], or random forests [9], among many others. Several previous works on this topic have used nonlinear set membership (NSM) methods [10] for learning, like [11,12]. In previous works, Email addresses: [email protected] (Jos´ e Mar´ ıa Manzano), [email protected] (Daniel Limon), [email protected] (David Mu˜ noz de la Pe˜ na), [email protected] (Jan-Peter Calliess). The authors thank the support by the MINECO-Spain and the authors proposed to use kinky inference methods [13] in a model predictive control setting. Kinky inference (KI) methods encompass Lipschitz interpolation [11, 14] and NSM methods [12], and they have several properties that are interesting for MPC. A modified version of this method, tailored to model predictive control, was proposed and used to design a MPC with guaranteed closed-loop properties for systems subject to input constraints [15]. The main limitation of this controller is that it cannot guarantee robust satisfaction of output constraints. In this paper, the main contribution is a new robust predictive controller for systems that also have output constraints. A design method that takes into account the prediction error bounds in an explicit way to tighten the problem constraints and that guarantees closed-loop constraint satisfaction and input-to-state stability (ISS) [16] is provided. One of the main characteristics of this design is that it is not based on a terminal region constraint. In general, this terminal constraint is based on robust invariant sets which, for the class of systems considered (that is, unknown systems, possibly nonlinear, for which a priori only input/output data is available), are difficult to obtain. Another contribution of this work is the procedure to define the tightened constraints of the MPC optimization problem. They are specifically tailored to the inference model used, in order to obtain the least conservative possible bounds. In contrast to the preliminary version of this controller, pre- FEDER Funds under project DPI2016-76493-C3-1-R, as well as the VI Plan PPIT-US of the University of Seville. sented in [17], an output-feedback formulation is considered, and the stability analysis is addressed. Notation: For two column vectors v, w,(v, w)implies [vT, wT]T. Given two sets A, B,A⊕Bis the Minkowski sum and ABthe Pontryagin difference. The set of the integers in the interval [a, b]is denoted Ib a. A function α:R≥0→R≥0is a K-function if it is continuous, strictly increasing and α(0) = 0. Given a vector v∈Rny,the ball B(v)⊂Rnyis defined as B(v) = {y:|ys| ≤ vs, s ∈Iny 1}; and |v|denotes the vector whose components are the absolute value of the components of v.Inis an identity matrix of size n. Given a compact set Ω,kΩkq= maxx∈Ωkxkq, for some norm k·kq. 2 Problem setting In this paper it is assumed that the system to be controlled is a sampled continuous-time system described by an a priori unknown discrete-time model, where y(k)∈Rnyis its measured output and u(k)∈Rnuis the control input. Both inputs and outputs are subject to hard constraints u(k)∈ U, y(k)∈ Y,(1) where both Uand Yare compact sets. It is assumed, without loss of generality, that the origin is the equilibrium point of the system where the plant must be stabilized. It is assumed that the only information available from the plant is historical data, containing a certain set of measured inputs and outputs trajectories, D. The objective of the paper is to design an output-feedback control law u(k) = κ(y(k); D),(2) such that from the data set Dand the current output measurement y(k), the control action is computed. It is desired to devise the control law such that the closed-loop system is asymptotically stable and that the constraints are satisfied for all time steps k∈N. Since a model of the dynamics is not available a priori, it is assumed that the measured output can be used to describe the model of the system with the following nonlinear autoregressive exogenous (NARX) model of the plant [18,19]: y(k+ 1) = f(x(k), u(k)) + e(k),(3) where x(k)=(y(k),· · · , y(k−na), u(k−1),· · · , u(k− nb)) ∈ X := Y(na+1) × Unb⊆Rnxwith nx= (na+ 1)ny+nbnu, for some memory horizon lengths na, nb∈N. The residual e(k)models process noise and it is assumed to be confined to a compact set E ⊂ Rny. For notational convenience, the inputs of fare aggregated into a joint vector w:= (x, u)∈ W, which is referred to as regressor. Remark 1 The horizons naand nbrepresent the model order. In [19], the conditions under which they could be taken as na=nb= 2nwere given, where nis the order of the system. If nis unknown, one would have to use the best guess, or cross-validation methods to estimate naand nb. Assumption 1 (H¨ older continuity) Each output component of the function f(·), referred to as ground truth function, is H¨ older continuous. That is, there exist some constants Lf,i >0and pf,i ∈(0,1] (i∈Iny 1) such that ∀w1, w2∈ W |fi(w1)−fi(w2)| ≤ Lf,ikw1−w2kpf,i W, i ∈Iny 1,(4) where k · kWstands for a specific norm defined for the regressors, and the sub-index idenotes the i-th component of the vector. For each i∈Iny 1, any constant Lf,i that satisfies this condition is called a H¨ older constant, while the lowest of them is called the best H¨ older constant, L∗ f,i. Remark 2 Assumption 1 can be relaxed to general continuity, provided that both Yand Uare compact sets. 2.1 The learning method In this section the machine learning method used to estimate fis presented, sometimes called kinky inference [13]. Using the available experimental data, a data set Dof ND regressor/outputs is collected; that is D:= {(y(j), w(j)) |j= 1,· · · , ND}.(5) The structure of Ddepends on the value of naand nb. To predict an unseen query point w, KI makes use of the data base D, and provides an estimation of the H¨ older parameters, denoted Land p, which are vectors of dimension ny.1 Definition 1 (Kinky inference rule ) The i-th output component function of the KI predictor, for i= 1, ..., ny, shall be defined by ˆ fiw;Li, pi,D:= 1 2min j=1,...,ND yj,i +Likw−wjkpi W +1 2max j=1,...,ND yj,i −Likw−wjkpi W.(6) Given the hyperparameters Land pand the data set D, the predictor ˆ fof the ground-truth fis constructed, yielding the output prediction: 2 ˆy(k+ 1) = ˆ f(x(k), u(k)).(7) 1In order to reduce the estimation error, different H¨ older constants Liare used to estimate the i-th entry of the output vector. With a slight abuse of notation Land pwill denote a vector in Rny. 2For the sake of conciseness, the dependence of ˆ fwith L,pand Dmay be omitted in the rest of the paper. 2 This predictor ˆ fis H¨ older continuous, and as proven in [20, Lemma 5], the H¨ older parameters of fare also H¨ older parameters of ˆ f. That is, for a given exponent p, the Lipschitz constant L∗is also a Lipschitz constant of ˆ f. However, the estimated Land pwill suffice to derive the stability properties of the proposed controller, as it will be explained later. Remark 3 While in several works a priori knowledge of the correct parameters Lfand pfis assumed [10, 21], other works provide methods of calculating these parameters from the available data [12, 20]. In this paper, the socalled LACKI method [20] is applied. It obtains the Lipschitz constant Las the minimum one that is consistent with the data. The learning feature of the proposed predictor is also demonstrated in [20]. Note that, w.l.o.g., this paper extends the LACKI method to each output component function in isolation. In this paper, the paradigm of predictive control will be employed to derive the data-based control law (2). MPC requires repeated optimization of the predicted control inputs subject to constraints. Therefore, in order to give guarantees on the controller’s closed-loop performance, recursive feasibility and constraint satisfaction must be ensured. That is, it is necessary to ensure that all constraints remain satisfiable during runtime, or equivalently, to guarantee that the controlled system will not leave the feasibility region. However, since the controller will not be based on the groundtruth dynamics f, but on the learned model ˆ finferred from a sample of the ground-truth, recursive feasibility can only be guaranteed if a bound on the discrepancy between fand ˆ fis known a priori and taken into account by the controller. The estimation method ensures that if the model function is H¨ older and the noise is bounded, then the estimation error is bounded [20], which is required to design a deterministic robust controller to regulate the plant. Any worst-case guarantee inevitably requires a priori knowledge. Hence, in the following hypothesis, it is assumed that this bound is available for the design of the controller. Assumption 2 It is assumed that for L,p, and D, a bound on the error between the estimated output and the real output is known, denoted µ∈Rny, such that |ˆ fi(x, u)−fi(x, u)−ei| ≤ µi,(8) for all i∈Iny 1,e∈ E,x∈ Yna+1 × Unb, and u∈ U. Remark 4 From a practical point of view the problem of how to calculate the error bound must be addressed. Kinky inference methods enjoy the property of providing a deterministic error bound if the Lipschitz constant and an upper bound of the noise are known [13]. Moreover, if a bound on the second derivative is known, it is also possible to derive an estimation error bound. If these parameters are not known a priori, which is usual in practice, then they must be estimated from experimental data. Consequently, the validity of the results presented in this paper is conditioned to the validity of the estimated error bound. This is the reason why this is considered as a standing assumption. Remark 5 The KI prediction method has recently been improved in [15] decreasing the computational cost and smoothing the prediction, in order to enhance the optimization that will be carried out by the controller. 3 Stabilizing data-based NMPC In this section, a model predictive controller is derived based on a prediction model learned from data of the plant. Since the prediction model is not accurate, the effect of the estimation error on the predictions must be analysed to be taken into account in the design of the controller. For this analysis, it is convenient to define the NARX model of the plant in a state-space form as follows: x(k+ 1) = F(x(k), u(k)) + ξ(k)(9a) y(k) = Mx(k),(9b) where F(x(k), u(k)) = f(x(k), u(k)), y(k),· · · , y(k−na+ 1), u(k),· · · , u(k−nb+ 1),(10) M= [Iny,0,· · · ,0], and ξ(k)=(e(k),0,· · · ,0). Let ˆy(j|k)denote the output that, at time k, is predicted to be observed at time k+j, for a given candidate control sequence u(k+j), j ∈IN−1 0. Then the predicted state is given by ˆx(j|k) = (ˆy(j|k),· · · ,ˆy(1|k), y(k),· · · , y(k+j−na), u(k+j−1),· · · , u(k+j−nb)), where ˆx(j+ 1|k) = ˆ F(ˆx(j|k), u(k+j)) (11) and ˆ F(ˆx(j|k), u(k+j)) = (ˆ f(ˆx(j|k), u(k+j)), ˆy(j|k),· · · , y(k),· · · , y(k+j−na+ 1),· · · , u(k+j),· · · , u(k+j−nb+ 1)). The proposed robust MPC is based on nominal predictions and tightened constraints. To guarantee robustness, a bound on the propagation of the prediction error is calculated from the following lemma: Lemma 1 Assume that at sampling time k, the state of the plant is x(k)and a sequence of future control inputs u(k+j) 3 y(k) y(k+ 1) y(k+j) ˆy(1|k) ˆy(j|k) ˆy(j−1|k+ 1) cj c1 Fig. 1. Propagation of the prediction error for j∈IN−1 0is given. Let ˆx(j|k)and ˆy(j|k)be the predicted states and outputs, respectively, derived from (11) for the given sequence of future control inputs and the current state x(k), i.e. ˆx(0|k) = x(k). Assume that at sampling time k+ 1, the current output y(k+ 1) is measured, and hence the current state x(k+ 1) is known. Based on these new measurements, an updated sequence of states and outputs ˆx(j|k+ 1) and ˆy(j|k+ 1) are predicted based on (11) with ˆx(0|k+1) = x(k+1) and the remaining sequence of the given future control inputs. Let c1∈Rnybe a vector such that |y(k+ 1) −ˆy(1|k)| ≤ c1.(12) Then, the mismatch between the predictions satisfies 3 |ˆy(j−1|k+ 1) −ˆy(j|k)| ≤ cj, j ∈IN 1,(13a) kˆx(j−1|k+ 1) −ˆx(j|k)kX≤rj, j ∈IN 1,(13b) where cj∈Rnyand rj∈Rare obtained from the recursion cj+1,i =Lirpi j,(14) and rj=kΞjkX,j∈IN−1 1,i∈Iny 1, where Ξj=B(cj)× · · · × B(cσ(j))× {0}×···×{0} | {z } nb+1−σ(j−1) times ⊆Rnx, with σ(j) = max(1, j −na). PROOF. Provided that ˆy(j−1|k+ 1) = ˆ f(x(j−2|k+ 1, u(k+j−1)) and ˆy(j|k)) = ˆ f(ˆx(j−1|k), u(k+j−1)), it can be derived that ∀i∈Iny 1 |ˆyi(j−1|k+ 1) −ˆyi(j|k))| ≤ Likˆx(j−2|k+ 1) −ˆx(j−1|k)kpi X. 3k·kXis a norm for the state-space such that kxkX=k(x, 0)kW Given that ˆx(j−2|k+ 1) −ˆx(j−1|k) = [ˆy(j−2|k+ 1) −ˆy(j−1|k),ˆy(j−3|k+ 1) −ˆy(j−2|k), . . . , ˆy(σ(j−1) −1|k+ 1) −ˆy(σ(j−1)|k),0,...,0] . then ˆx(j−2|k+ 1) −ˆx(j−1|k)∈Ξj−1. Assuming that cj−1is known, kˆx(j−2|k+ 1) −ˆx(j−1|k)kX≤ kΞj−1kX=rj−1, which implies the stated result. Remark 6 If the infinity norm is chosen as the norm of the input space then rj= max s∈Ij σ(j) kcsk∞. Based on the derived bounds on the prediction error, the problem of robust constraint satisfaction is addressed by means of a set of tightened constraints on the outputs [22], computed offline for the maximum possible prediction error, i.e. taking c1=µ. These sets are defined as follows: Yj=Y  B(dj),(15) where dj= j X s=1 cs.(16) These constraints sets will be used to prove recursive feasibility of the controller, following standard procedures. Lemma 2 The sets Yjare such that for all y∈ Yjand for all ∆y∈ B(cj),y+ ∆y∈ Yj−1. PROOF. Since for j≥1,dj=dj−1+cj, it follows that B(dj) = B(dj−1)⊕ B(cj). By definition, y+ ∆y∈ Yj⊕ B(cj) = Y  B(dj)⊕ B(cj), and hence Yj=Y  B(dj) = Y  B(dj−1) B(cj), so y+ ∆y∈ Yj⊕ B(cj) =Y  B(dj−1) B(cj)⊕ B(cj) ⊆ Y  B(dj−1) = Yj−1. In order to ensure that the proposed controller is feasible, the tightened set of constraints must be non-empty along the prediction horizon, as stated in the following assumption: Assumption 3 The prediction horizon Nand the estimation error bound µare such that the set YNis non-empty. 4 Based on the previous definitions, the optimization problem PN(x(k); D)of the proposed predictive controller is: min u VN(x(k),u) = N−1 X i=0 `(ˆx(i|k), u(i)) + λVf(ˆx(N|k)) (17a) s.t. ˆx(0|k) = x(k)(17b) ˆx(j+ 1|k) = ˆ F(ˆx(j|k), u(j)), j ∈IN−1 0(17c) ˆy(j|k) = Mˆx(j|k), j ∈IN−1 0(17d) u(j)∈ U, j ∈IN−1 0(17e) ˆy(j|k)∈ Yj, j ∈IN−1 0,(17f) where λ≥1is a weighting parameter. Note that this problem is non-linear, non-convex and non-differentiable. Its ingredients are required to meet the following assumption, which is similar to the standard MPC ones [22]: Assumption 4 (1) The stage cost function `(·,·)is a H¨ older continuous positive definite function such that `(·,·)≥αy(kxkX) for a certain K-function αy, and its H¨ older parameters are Lxand px. (2) There exists a control law u=κf(x), a function Vf and a level set Ωγ={x:Vf(x)≤γ} ⊆ Rnxfor some γ > 0such that for all x∈Ωγthe following conditions hold: (a) Vfis a H¨ older continuous positive definite function, with H¨ older constants LVf, pVf, such that αf(kxkX)≤Vf(x)≤βf(kxkX), Vf(ˆ F(x, κf(x))) −Vf(x)≤ −`(x, κf(x)). (b) κf(x)∈ U,Mx ∈ YN. The controller is derived from the receding horizon solution of (17). It follows a standard robust approach in which the cost of the nominal predictions is minimized, while taking into account a tightened set of constraints to guarantee recursive feasibility. The main difference with off-the-shelf robust ISS formulations for nonlinear systems [16] is that in these, either there are no constraints on the states in the optimization problem, or a terminal constraint, based on a certain robust positive invariant set, is added. In this controller, although a terminal cost (based on a local controller for the nominal model) is taken into account in the cost function, no terminal constraint is included. Thus, its design is notably simplified since the calculation of a robust invariant set is avoided, which was a hard task, as shown in [23]. In this case, the calculation could have been even more difficult provided the lack of a explicit expression of the model of the system. Furthermore, an additional tuning parameter λis added, modifying the weight of the terminal cost in the objective function. It is proven that this controller guarantees that the closed-loop systems is ISS in a explicitly defined region of the state space, which is enlarged by this weight. Define the function ν(c1) = N X j=1 Lxrpx j+λLVfrpVf N+1, where rjis defined in Lemma 1 for c1, and Lx, LVfin Ass. 4. Assumption 5 The bound µis such that the set Υ = {x:`(x, 0) ≤ν(µ)} is contained in Ωγ. The positive constants λand φare such that λ≥1and `(x, 0) > φ for all x6∈ Ωγ. Remark 7 In a general setting, a condition to check if the level set Υis contained in Ωγcould be derived using the supply K∞-functions that bound the cost functions given in Assumption 4. In this case the condition would be: ν(µ)≤αy(β−1 f(γ)).(18) Another method could be using probabilistic validation by means of randomized algorithms [24]. Lemma 3 Under Assumption 5, φ≥ν(µ). PROOF. Since Υ⊆Ωγ, the constant φcan be taken as φ≥minx∈X\Ωγ`(x, 0) ≥minx∈X \Υ`(x, 0) ≥ν(µ). Let Γdefine the following level set of the optimal cost function Γ = {x:V∗ N(x)≤Nφ +λγ}. It is next proven that this set defines the region in which ISS is guaranteed. Notice that this set is compact and non-empty. Theorem 1 (ISS stability) Suppose that Assumptions 2, 3, 4 and 5 hold for the optimization problem PN(·). Let κN(x) be the control law derived from the solution of PN(x;D)applied using a receding horizon policy. Then, for any x(0) ∈ Γ, the system controlled by the control law u(k) = κN(x(k)) is input-to-state stable w.r.t. the estimation error; and the constraints are always satisfied, i.e. u(k)∈ U,y(k)∈ Y and x(k)∈Γ,∀k. PROOF. Assume that x(k)∈Γ. Then, it can be shown that x∗(N|k)∈Ωγ[25]. Define the shifted sequence as ¯ u(k+ 1) such that ¯u(j|k+ 1) = u∗(j+ 1|k)for j∈IN−2 0 and ¯u(N−1|k+ 1) = κf(x∗(N|k)). Recursive feasibility: Assuming that x(k)∈Γ, it will be proven that x(k+1) ∈Γ. Since Γis a subset of the feasibility region of the optimization problem PN(x;D), the system is recursively feasible. Firstly, it will be shown that the solution ¯ u(k+1) is a feasible solution for x(k+ 1). Given that x∗(N|k)∈Ωγ, from the 5 feasibility of u∗(k)and assumption 4, it is immediate to state that ¯u(j|k+ 1) ∈ U for all j∈IN−1 0. From Lemma 1, ˆy(j|k+ 1) −y∗(j+ 1|k)∈ B(cj+1),∀j∈ IN−1 0and from the feasibility of u∗(k). Thus, y∗(s|k)∈ Ys for s∈IN−1 0and x∗(N|k)∈Ωλ, which implies that y∗(N|k) = Mx∗(N|k)∈ YNin virtue of assumption 4.2b. Then, from Lemma 2, for all j∈IN−1 0, ˆy(j|k+ 1) ∈ Yj+1 ⊕ B(cj+1)⊆ Yj. Therefore the problem PN(x(k+ 1); D)is feasible. Next, it will be proven that x(k+ 1) ∈Γ. Since x(k)∈Γ, V∗ N(x(k)) ≤Nφ +λγ. Following standard arguments in MPC [22] it can be proven that VN(x∗(1|k),¯ u(k+ 1)) ≤V∗ N(x(k)) −`(x(k), u(k)) ≤Nφ +λγ −`(x(k), u(k)). On the other hand, given ˜x=ˆ F(x∗(N|k), κf(x∗(N|k)), VN(x(k+ 1),¯ u(k+ 1)) −VN(x∗(1|k),¯ u(k+ 1)) = N−1 X i=0 `(ˆx(i|k+ 1),¯u(i|k+ 1)) −`(x∗(i+ 1|k),¯u(i|k+ 1)) +λVf(ˆx(N|k+ 1)) −Vf(˜x). According to Lemma 1, it can be derived that for s∈IN 1, kˆx(s−1|k+ 1) −x(s|k)kX≤rs, with rsobtained for a given c1. Then, for j∈IN 1and given c1satisfying (12), `(ˆx(j−1|k+ 1),¯u(j−1|k+ 1)) −`(x∗(j|k),¯u(j−1|k+ 1)) ≤Lxrpx j and Vf(ˆx(N|k+ 1)) −Vf(˜x)≤LVfrpVf N+1. Therefore, VN(x(k+ 1),¯ u(k+ 1)) −VN(x∗(1|k),¯ u(k+ 1)) ≤ν(c1). (19) To prove robust invariance the worst possible case has to be considered, for which c1=µis taken. Hence, it has been proven that VN(x(k+1),¯ u(k+1)) ≤ν(µ)+Nφ+λγ −`(x(k), u(k)). Consider the case where x(k)∈Γ\Υ. Then `(x(k), u(k)) > ν(µ). Hence, VN(x(k+1),¯ u(k+1)) ≤ν(µ)+Nφ +λγ − `(x(k), u(k)) ≤Nφ +λγ. Given that V∗ N(x(k+ 1)) ≤VN(x(k+ 1),¯ u(k+ 1)) then x(k+ 1) ∈Γ. Consider now the case that x(k)∈Υ. Since Υ⊆Ωγ, x(k)∈Ωγ. With standard arguments in MPC [22], it can be shown that V∗ N(x(k)) ≤λVf(x(k)) ≤λγ. Hence, VN(x(k+ 1),¯ u(k+ 1)) ≤ν(µ) + V∗ N(x(k)) −`(x(k), u(k)) ≤ν(µ) + λγ −`(x(k), u(k)), since ν(µ)≤φ,VN(x(k+1),¯ u(k+1)) ≤Nφ+λγ. Thus, x(k+ 1) ∈Γ. Input-to-state stability: Equation (19) can be rewritten as follows, taking c1=ep(k+ 1) := |y(k+ 1) −ˆy(1|k)|: VN(x(k+ 1),¯ u(k+ 1)) −VN(x∗(1|k),¯ u(k+ 1)) ≤ν(ep(k+ 1)).(20) Then, following the previous steps, it can be derived that V∗ N(x+ 1) ≤VN(x(k+ 1),¯ u(k+ 1)) (21) ≤ν(ep(k+ 1)) + V∗ N(x(k)) −`(x(k), u(k)). Thus, V∗ N(x)is an ISS Lyapunov function [16]. Remark 8 (Suboptimal case) The stability analysis can be extended to the case in which the optimal solution of the control problem is not found. Given an initial feasible solution of the control problem, if the optimiser is able to improve the cost (even for a suboptimal solution of the problem), then the controller is able to maintain robust stability while satisfying the constraints [22]. Remark 9 (Violation of Assumption 2) If the bound on the prediction error is estimated from data (e.g. via crossvalidation), then the error could take a value larger than µ for a certain period of time. In this case, the ISS property (21) still holds as long as x(k)∈Γfor that period of time. Notice that the ISS condition is derived from the smoothness of the optimal cost function, which is an inherent property of the proposed optimal control problem. Remark 10 (Stability margin) Most of the robust controllers for constrained systems exhibit an upper bound on the estimation error to be solvable [16]. This is the socalled stability margin. One of the main drawbacks of robust predictive controllers is that this margin is typically quite conservative, due to the open-loop nature of the predictions. As a robust controller, our approach inherits this drawback. 4 Case study In this section, the proposed controller is applied to a continuously-stirred tank reactor [15], in which a reaction A→Btakes place. The system’s state is defined by the concentration of the reactant, CA(mol/l), the temperature in the tank, T(K), and the temperature of the coolant, Tc(K). The state varies with respect to the control input, 6 50 100 150 200 250 0.4 0.6 0.8 1 50 100 150 200 250 280 290 300 310 Fig. 2. Simulation obtained applying a sequence of chirp signals which is the reference temperature of the coolant, Tr(K), according to the following set of ordinary differential equations (ODEs), which will be used to simulate the system but are assumed to be unknown: dCA(t) dt =q0 V·(CAf −CA(t)) −k0·e−E R·T(t)·CA(t),(22a) dT(t) dt =q0 V·(Tf−T(t)) +(−∆Hr)·k0 ρ·Cp ·e−E R·T(t)·CA(t) +U·A V·ρ·Cp ·(Tc(t)−T(t)),(22b) dTc(t) dt =Tr(t)−Tc(t) τ.(22c) The parameters of the system can be found in [26], with τ= 1.5 min. The sampling time is 30 s. The output is CA, and its sensor has a 2.5% error margin, generated randomly using an uniform distribution. The constraints are given by 0.38 ≤CA≤0.954 mol/land 280 ≤Tr≤310 K. The reference equilibrium point is given by Cref A= 0.62 mol/l and Tref r= 304.5 K. In order to identify the system, several data sets are generated. The training data set is obtained using the following input sequence of chirp signals: five chirp signals of length 1000 min, initial and final frequencies of 1 mHz and 0.15 Hz (respectively), amplitude 5 K, and centres starting from 285 K with 5 K interval; followed by two chirp signals of length 5000 min, centered in 295 K,15 K of amplitude, and initial and final frequencies of 10 and 200 mHz and 1and 90 mHz, respectively, as represented in Figure 2. Another data set is obtained to calculate µvia crossvalidation, applying a pseudorandom binary input sequence, where Trswitches randomly between 280 and 310 K with switching periods between 12 and 75 min. Following standard cross-validation procedures [27], these data sets are used to define a predictor for different values of the memory horizons naand nb. Setting the prediction horizon to N= 4 and applying LACKI [20] to calculate Lwhile 0 2 4 6 8 10 12 0.4 0.6 0.8 1 0 2 4 6 8 10 12 0.4 0.6 0.8 1 0 2 4 6 8 10 12 0.4 0.6 0.8 1 Fig. 3. Closed-loop output for 100 simulations of the ideal MPC (top), the robust learning MPC for systems without output constraints (middle) and the proposed robust constrained learning MPC (bottom). The grey band groups the trajectories, the blue dashed line represents its mean, the green dotted one the reference and the black dash-dotted one the constraints fixing p= 1, the optimal na, nbare obtained minimising d4, which is calculated using (16). The minimum is obtained for na=nb= 2, for which L= 1.38,µ= 0.032 mol/land d4= 0.22 mol/l. Both the stage and the terminal cost of the MPC are defined as follows, with xref = (yref, . . . , yref, uref, . . . , uref): `(x, u) = kx−xrefk2 Q+ku−urefk2 R,(23a) Vf(x) = kx−xrefk2 P.(23b) Qis set to 100,R= 0.1, and λ= 10. Using the model with na=nb= 2 the terminal cost is obtained solving a LQR for the linearised model around the reference point. To ensure robust stability (Theorem 1), Assumptions 2, 3, 4 and 5 must hold true. The prediction error was obtained via cross-validation. The value of dNresults in YN={y: 0.60 ≤y≤0.73}. Following the procedure in section 3 and in [15] results in γ= 16796,ν(µ) = 53.553 and φ= 9.2189 ×105, which satisfy all the assumptions. The proposed controller is applied in 100 closed-loop simulations, subject to random noise. The results are shown in the last row of Figure 3. Note that the output is steered to the reference while the constraints are satisfied. The optimization problem is solved in Matlab on an Intel® Core™ i7-6700HQ CPU @ 2.60 GHz 12GB RAM and each iteration takes less than one second to complete, much shorter than the 30 s required by the sampling time. 7 In order to compare the proposed controller to other methods, the same setup is simulated with two different MPCs. First, a controller derived from (17), but with the set of ODEs (22) as the state-feedback prediction model, for which µis the maximum noise, 0.025 mol/l. This aims to resemble the ideal case of perfect knowledge of the plant (result shown in the first row of Fig. 3). As expected, the data-based controller performs slower than the ideal since, unlike the latter, an output-feedback uncertain framework is considered. Second, the controller proposed in [15] is applied. It is based on KI and guarantees closed-loop stability but does not take into account output constraints, so as shown in the second row of Fig. 3, the minimum CAlimit is violated. To sum up, the controller proposed in this paper is able to robustly satisfy hard constraints in the outputs, learning the model from input-output data with a closed-loop performance similar to the ideal case. 5 Conclusion A novel learning-based predictive controller capable of ensuring robust stability without terminal constraint was proposed and proven to be robustly stable (in the ISS sense) and recursively feasible under some assumptions. The plant model of this controller is learned from input-output data, using the LACKI approach. Under assumptions on conservatism of estimated maximum bounds, tight bounds on the effect of the multiple step look-ahead uncertainty were derived and ISS stability was proven. Simulation of a continuously-stirred tank reactor illustrated the practical feasibility of the proposed controller. References [1] F. Allg¨ ower and A. Zheng, Nonlinear model predictive control. Birkh¨ auser, 2012, vol. 26. [2] M. Behl, A. Jain, and R. Mangharam, “Data-driven modeling, control and tools for cyber-physical energy systems,” in Proceedings of the 7th International Conference on Cyber-Physical Systems. IEEE Press, 2016, p. 35. [3] L. Hewing, K. P. Wabersich, M. Menner, and M. N. Zeilinger, “Learning-based model predictive control: Toward safe learning in control,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 3, 2019. [4] A. Aswani, H. Gonzalez, S. S. Sastry, and C. Tomlin, “Provably safe and robust learning-based model predictive control,” Automatica, vol. 49, no. 5, pp. 1216–1226, 2013. [5] D. Piga, S. Formentin, and A. Bemporad, “Direct data-driven control of constrained systems,” IEEE Transactions on Control Systems Technology, vol. 26, no. 4, pp. 1422–1429, 2017. [6] J. R. Salvador, D. R. Ramirez, T. Alamo, and D. Mu˜ noz de la Pe˜ na, “Offset free data driven control: application to a process control trainer,” IET Control Theory & Applications, vol. 13, no. 18, pp. 3096–3106, 2019. [7] J. F. Fisac, A. K. Akametalu, M. N. Zeilinger, S. Kaynama, J. Gillula, and C. J. Tomlin, “A general safety framework for learningbased control in uncertain robotic systems,” IEEE Transactions on Automatic Control, vol. 64, no. 7, pp. 2737–2752, 2018. [8] M. Maiworm, D. Limon, J. M. Manzano, and R. Findeisen, “Stability of gaussian process learning based output feedback model predictive control,” IFAC-PapersOnLine, vol. 51, no. 20, pp. 455–461, 2018. [9] F. Smarra, A. Jain, T. De Rubeis, D. Ambrosini, A. Dˆ aInnocenzo, and R. Mangharam, “Data-driven model predictive control using random forests for building energy optimization and climate control,” Applied energy, vol. 226, pp. 1252–1272, 2018. [10] M. Canale, L. Fagiano, and M. C. Signorile, “Nonlinear model predictive control from data: a set membership approach,” International Journal of Robust and Nonlinear Control, vol. 24, no. 1, pp. 123–139, 2014. [11] A. Sukharev, “Optimal method of constructing best uniform approximations for functions of a certain class,” USSR Computational Mathematics and Mathematical Physics, vol. 18, no. 2, pp. 21–31, 1978. [12] M. Milanese and C. Novara, “Set membership identification of nonlinear systems,” Automatica, vol. 40, no. 6, pp. 957–975, 2004. [13] J.-P. Calliess, “Conservative decision-making and inference in uncertain dynamical systems,” Ph.D. dissertation, University of Oxford, 2014. [14] G. Beliakov, “Interpolation of lipschitz functions,” Journal of computational and applied mathematics, vol. 196, no. 1, pp. 20–44, 2006. [15] J. M. Manzano, D. Limon, D. Mu˜ noz de la Pe˜ na, and J. P. Calliess, “Output feedback MPC based on smoothed projected kinky inference,” IET Control Theory & Applications, vol. 13, no. 6, pp. 795–805, 2019. [16] D. Limon, T. Alamo, D. M. Raimondo, D. Mu˜ noz de la Pe˜ na, J. M. Bravo, and E. F. Camacho, “Input-to-state stability: an unifying framework for robust model predictive control,” in Proceedings of the Int. Workshop on Assessment and Future Directions of Nonlinear Model Predictive Control (NMPC08). 5-9 Sept. Pavia, Italy., 2008. [17] J. M. Manzano, D. Limon, D. Mu˜ noz de la Pe˜ na, and J. P. Calliess, “Robust data-based model predictive control for nonlinear constrained systems,” IFAC-PapersOnLine, vol. 51, no. 20, pp. 505–510, 2018. [18] I. Leontaritis and S. A. Billings, “Input-output parametric models for non-linear systems part i: deterministic non-linear systems,” International journal of control, vol. 41, no. 2, pp. 303–328, 1985. [19] A. Levin and K. Narendra, “Identification of nonlinear dynamical systems using neural networks,” in Neural systems for control. Elsevier, 1997, pp. 129–160. [20] J.-P. Calliess, “Lazily adapted constant kinky inference for nonparametric regression and model-reference adaptive control,” arXiv preprint arXiv:1701.00178, 2016. [21] Z. B. Zabinsky, R. L. Smith, and B. P. Kristinsdottir, “Optimal estimation of univariate black-box lipschitz functions with upper and lower error bounds,” Computers & Operations Research, vol. 30, no. 10, pp. 1539–1553, 2003. [22] J. B. Rawlings and D. Q. Mayne, Model Predictive Control: Theory and Design, 1st ed. Nob-Hill Publishing, 2009. [23] M. Fiacchini, T. Alamo, and E. F. Camacho, “On the computation of convex robust control invariant sets for nonlinear systems,” Automatica, vol. 46, no. 8, pp. 1334–1338, 2010. [24] G. Calafiore and F. Dabbene, Probabilistic and randomized methods for design under uncertainty. Springer, 2006. [25] D. Limon, T. Alamo, F. Salas, and E. F. Camacho, “On the stability of MPC without terminal constraint,” IEEE Transactions on Automatic Control, vol. 42, pp. 832–836, 2006. [26] L. Magni, G. De Nicolao, L. Magnani, and R. Scattolini, “A stabilizing model-based predictive control algorithm for nonlinear systems,” Automatica, vol. 37, pp. 1351–1362, 2001. [27] L. Ljung, System identification. Springer, 1998. 8