A robust optimal control approach in the weighted Sobolev space for underactuated mechanical systems

Depósito de Investigación de la Universidad de Sevilla https://idus.us.es/ This is an Accepted Manuscript of an article published by Elsevier in Automatica, Vol. 125, on March 2021, available at: https://doi.org/10.1016/j.automatica.2020.109474 © 2021 Elsevier. En idUS Licencia Creative Commons CC BY-NC-ND A robust optimal control approach in the weighted Sobolev space for underactuated mechanical systems? Daniel Neri Cardosoa, Sergio Estebanb, Guilherme V. Raffoa,c aGraduate Program in Electrical Engineering, Federal University of Minas Gerais, 31270-901, Belo Horizonte, MG, Brazil. bDepartment of Aerospace Engineering, Universidad de Sevilla, 41092, Seville, Spain. cDepartment of Electronic Engineering, Federal University of Minas Gerais, Belo Horizonte, MG, Brazil. Abstract This paper proposes new robust nonlinear optimal control formulations in the weighted Sobolev space, here called nonlinear W2 and W∞controllers, to handle with two classes of underactuated mechanical systems: the reduced ones with a reduced number of DOF; and the entire underactuated mechanical systems with input coupling. The optimal control problems are formulated via dynamic programming and particular solutions are presented for the resulting Hamilton-Jacobi equations with the corresponding stability analysis. Also, the concepts of Wm,p,σ-stability and Wm,p,σ-gain for a general class of systems are established, with the demonstration for the particular case of study. The novel W2and W∞controllers are synthesized for an unmanned aerial vehicle benchmark. The results demonstrate that these controllers provide better transient performance with faster response against external disturbances in comparison with a classic nonlinear H∞controller, in addition to have a simple design. Keywords: Nonlinear control, W2control, W∞control, Robust control, Sobolev space, Underactuated mechanical systems. 1. Introduction Control of mechanical systems is currently among the most active fields of research due to the diverse academic and industrial applications in real-life, for example, robots, aircraft, automobiles, and cranes. Commonly, the control design for these5 kind of systems is based on mathematical models; however, in general, these models present errors due to unmodeled dynamics and parameter estimation. In addition, mechanical systems are generally subjected to noise added to the measurement reported by sensors and external disturbances. The inevitable10 presence of uncertainties in mechanical systems requires the design of robust controllers to achieve acceptable performance. Two usual approaches to deal with disturbances and imperfections on the mathematical model in the control design stage are the well known H2[1] and H∞[2] control strategies.15 They have been originally formulated in the frequency domain to cope with single-input-single-output systems represented by transfer functions [3]. The H2controller designed on the frequency domain seeks to minimize the energy of the disturbance impulse response [4], while the H∞controller minimizes the20 maximum gain given by the closed-loop system to a disturbance signal [5]. Due to the need to deal with multiple-input-multiple- output systems, efforts were made to extend them to systems ?This work was supported by the Brazilian agencies CNPq, CAPES, FAPEMIG, and FAPESP. The authors would also like to acknowledge the INCT (National Institute of Science and Technology) for Cooperative Autonomous Systems Applied to Security and Environment under the grant CNPq 465755/2014-3, FAPESP 2014/50851-0. Email addresses: [email protected] (Daniel Neri Cardoso), [email protected] (Sergio Esteban), [email protected] (Guilherme V. Raffo) represented in the state-space [6] and was an initial point to the development of H2and H∞controllers based on linear matrix25 inequalities [7]. For nonlinear systems, the H∞control strategy was first introduced in [8], where the problem is formulated in the L2space using the theory of dissipative systems. Concerning the development of H2and H∞controllers for mechanical systems, a pioneering work is [1] where the nonlin-30 ear H2control problem is formulated using dynamic programming for the class of fully-actuated mechanical systems. The control problem results in solving a Hamilton-Jacobi (HJ) firstorder partial differential equation (PDE), for which a particular solution is presented. Of note, HJ PDEs have proven to be35 analytically impossible to solve for a general class of systems [9]. Based on the mentioned particular solution, [10] formulated the nonlinear H∞control strategy for the same class of systems. In the same year, [11] proposed a similar approach for robotic systems, where the cost variable considers the coupling40 between the controlled variables and the control inputs, giving extra degrees of freedom (DOF) to the control tuning. Although, the nonlinear H2and H∞control strategies have been applied successfully to mechanical systems in several experiments, these methods may present drawbacks. For exam-45 ple, as stated in [10], the standard formulation of the nonlinear H∞control and, consequently, of the nonlinear H2control for Euler-Lagrange mechanical systems presents a limitation in the way to weight the cost variable. For its appropriate formulation, the weighting matrices must be considered like positive50 real scalars multiplied by the identity matrix, limiting the adjustment of the control law for systems with multiple DOF with similar dynamics. In addition, as stated in [12], the H∞control strategy deals mostly with the aspect of the highest gain that Preprint submitted to Automatica January 23, 2025 the system gives to the disturbances, and provides little control55 over the transient behavior. Regarding the first drawback, in [13] an approach based on diagonalization of the inertia matrix is proposed in order to give flexibility to tune the nonlinear H∞ controller. On the other hand, few works deal with the second drawback for the control design of nonlinear systems.60 An alternative approach to overcome these issues is the formulation of both H2and H∞controllers in the Sobolev space Wm,p(the space of functions in the Lpspace whose generalized derivatives up to order mare also in the Lpspace) [14]. In the control area, Sobolev space properties have been successfully65 employed, for example, to design state observers [15, 16]. Furthermore, in [17] a new input-output stability is defined based on the Sobolev space Wm,p, as a local version of the small gain theorem, besides the relationship between W-stability and asymptotic stability. Also, [18] extended the W-stability cri-70 terion for the class of nonlinear systems with nonlinear finite time-delay. The idea behind these works relies on the fact that in this kind of space all properties of Lpspace are satisfied. In this context, [19] proposed the reformulation of the H2 and H∞control approaches in the Sobolev space to achieve75 improved transient performance. In their formulation, instead of the L2-norm, the W1,2-norm of the cost variable is considered.1The control problems are developed via dynamic programming, from which the related HJ equations must be solved to obtain the resulting controllers. For full-state feedback lin-80 ear systems, these controllers are designed by solving a matrix Riccati equation, for which efficient numerical algorithms can be used to achieve the solution [20]. Nevertheless, for nonlinear systems, the resulting HJ equations are hard to solve analytically, which, from our best knowledge, is an open field for85 research. In particular, [19] considers the HJ equation resulting from the H∞control formulation in the Sobolev space to be very hard to compute the solution, proposing an alternative approach through the backstepping technique to simplify the problem. However, the latter approach is similar to the clas-90 sic nonlinear H∞one, differing by an integrator added to the cost variable. Thus, as the rate of change of the cost variable is not considered in the cost functional, the improvements in the transient behavior are not achieved. In addition, the approaches presented in [19] lack to distinguish the influences of the cost95 variable and its time derivatives in the control objectives. Therefore, to deal with mechanical systems, this work proposes new formulations of the nonlinear H2and H∞control approaches in the weighted Sobolev space [21], which consider the weighted Sobolev norm2of the cost variable. Consequently,100 the influence of the cost variable and its time derivatives are 1The Sobolev Wm,p−norm of a function z(t) : R+→R, for m∈Nand p∈N∪ {∞}, is defined as ||z(t)||Wm,p=m P α=0 || dαz(t) dtα||p Lp1/p, where ||·||Lpstands for the Lp−norm, N={1,2,3...}and R+=[0,∞). 2The weighted Sobolev Wm,p,σ−norm of a function z(t) : R+→R, for m∈Nand p∈N∪ {∞}, is defined as ||z(t)||Wm,p,σ=m P α=0 || dαz(t) dtα||p Lp,σα1/p, where σ=(σ0, ..., σm), with σαbeing a symmetric, positive definite weighting matrix with appropriate dimension, and || · ||Lp,σαstands for the Lp−norm weighted by the matrix σα. weighted component-wise in the cost functional. Particular choices of the cost variable are proposed, such that the HJ equations resulting from the problems become more tractable allowing us to propose particular solutions. Additionally, aiming to105 improve the closed-loop performance with robustness against parametric uncertainties and constant disturbances, besides to attenuate disturbances in L2space, an integral action is considered in the problem formulation. The control approaches proposed in this paper are designed110 for the class of underactuated mechanical systems represented by Euler-Lagrange equations. When dealing with these kind of systems, due to the inability to perform simultaneous trajectory tracking of all DOF, it is common to manipulate the mathematical model by rewriting it as a reduced system [22, 23]. This115 reduced system considers the dynamics of controlled DOF and assumes the remaining ones having stable internal dynamics or are input to state stable with respect to the controlled DOF. Accordingly, initially we propose a methodology to synthesize the nonlinear W2and W∞controllers for reduced underactuated120 mechanical systems.3On the other side, if the system does not present stable zero dynamics or it is not intended to design an additional control strategy, the controller must guide the zero dynamics to a stable condition. Thus, aiming to drive the controlled DOF through a desired trajectory, while stabilizing the125 remaining ones, we also propose a methodology to design the nonlinear W2and W∞controllers for underactuated mechanical systems with input coupling [24, 25]. Summarizing, the main contributions of this work are: the proposal of the nonlinear W2and W∞control problems and130 their formulation via dynamic programming for (i) reduced underactuated mechanical systems and (ii) underactuated mechanical systems with input coupling; (iii,iv) the analytical solutions for the resulting HJ equations with the corresponding stability analysis; (v) the establishment of the concepts of Wm,p,σ-135 stability and Wm,p,σ-gain for a general class of systems, with the demonstration for the particular case of study; (vi) the design of the nonlinear W2and W∞controllers for a quadrotor unmanned aerial vehicle (UAV); and (vii) a comparative analysis of the results with a classic nonlinear H∞controller.140 2. Nonlinear W2and W∞control of reduced underactuated mechanical systems In this section the nonlinear W2and W∞optimal controllers are formulated for the class of reduced underactuated mechanical systems. The development is conducted considering systems represented by the Euler-Lagrange equations [26] M(q)¨ q+C(q,˙ q)˙ q+g(q)=B(q)τ+w(t),(1) where q∈Rnis the vector of generalized coordinates, M(q)∈ Rn×nthe symmetric, positive definite inertia matrix, C(q,˙ q)∈ Rn×nthe Coriolis and centripetal forces matrix, g(q)∈Rnthe145 gravitational force vector, B(q)∈Rn×cthe input coupling matrix with rank(B(q)) =c,τ(t)∈Rcthe control inputs, and 3Throughout the text the proposed H2and H∞control approaches formulated in Sobolev spaces will be referred as W2and W∞ones, respectively. 2 w(t)∈Rnthe generalized disturbances. It is assumed that c≤n and the generalized coordinates and velocities are measured. Moreover, it is well known that when dealing with underactuated mechanical systems no more than cDOF can be controlled (i.e. regulated at a desired position or track a reference trajectory) simultaneously through the control inputs, since the system has fewer manipulated variables than DOF. Then, due to the inability to perform simultaneous tracking of all DOF, for control design purposes, the system (1) is partitioned in controlled and uncontrolled DOF as follows "Muu(q)Muc(q) Mcu(q)Mcc(q)# | {z } M(q) "¨ qu ¨ qc# |{z} ¨ q +"Cuu(q,˙ q)Cuc(q,˙ q) Ccu(q,˙ q)Ccc(q,˙ q)# | {z } C(q,˙ q) "˙ qu ˙ qc# |{z} ˙ q (2) +"gu(q) gc(q)# | {z } g(q) ="Bu(q) Bc(q)# | {z } B(q) τ+"wu(t) wc(t)# | {z } w(t) , in which q=[q0 uq0 c]0, where (·)0denotes transpose, qu∈Rn−c corresponds to the uncontrolled DOF and qc∈Rcto the controlled ones. Considering this partition, it is possible to deal only with the controlled DOF. Therefore, by replacing ¨ quin the second row of (2), the controlled DOF dynamics are written as4 ¯ M(q)¨ qc+¯ C(q,˙ q)˙ qc+¯ e(q,˙ q)=¯ B(q)τ+¯ δ(q,t) (3) where ¯ M(q),Mcc −Mcu M−1 uu Muc,¯ C(q,˙ q),Ccc −Mcu M−1 uu Cuc,150 ¯ e(q,˙ q),gc−Mcu M−1 uu gu+Ccu −Mcu M−1 uu Cuu˙ qu,¯ B(q)=Bc− Mcu M−1 uu Bu,and ¯ δ(q,t),wc+Mcu M−1 uu wu. Accordingly, the system (2) must be partitioned such that the following assumption holds for the reduced system (3). Assumption 1. The matrix ¯ B(q)is invertible, i.e. rank(¯ B(q)) =155 c,∀q∈Ω, in which Ω⊆Rnis the configuration domain of the system. Remark 1. Regarding a fully-actuated mechanical system, i.e. τ(t)∈Rn,qc=q, yielding ¯ M(q)=M(q),¯ C(q,˙ q)=C(q,˙ q), ¯ e(q,˙ q)=g(q),¯ B(q)=B(q), and ¯ δ(q,t)=w(t). Consequently, (3)160 becomes equal to (1). Then, the approaches presented in this section are also valid for fully-actuated mechanical systems. Additionally, aiming to perform trajectory tracking, the error dynamics are written as ¯ M(˜qc+qcr,qu)¨ ˜ qc+¨ qcr+¯ C(˜qc+qcr,˙ ˜qc+˙ qcr,qu,˙ qu)˙ ˜ qc+˙ qcr(4) +¯ e(˜qc+qcr,˙ ˜qc+˙ qcr,qu,˙ qu)=¯ B(˜qc+qcr)τ+¯ δ(q,t), with ˜ qc,qc−qcr(t), and qcr(t), ˙ qcr(t) and ¨ qcr(t) being the desired values of the controlled DOF and their time derivatives, in which qcr(t)∈ C2. Besides, given the state vector x,˙ ˜ q0 c˜ q0 cRt 0˜ q0 cdt0, in which an integral action is considered over the error of the controlled DOF to provide parametric uncertainty and constant disturbance rejection capability for the closed-loop system, equation (4) can be represented in the state-space form as ˙ x= −¯ M−1¯ C0 0 I0 0 0I0  x | {z } f(x,qu,˙ qu) + −¯ M−1¯ d 0 0  | {z } ¯ f(x,qu,˙ qu,t) + ¯ M−1¯ B 0 0  | {z } g(x,qu) τ+ ¯ M−1 0 0  |{z} k(x,qu) ¯ δ,(5) 4For the sake of simplicity, throughout the text some function dependencies are omitted. where ¯ d,¯ M(˜qc+qcr,qu)¨ qcr(t)+¯ C(˜qc+qcr,˙ ˜qc+˙ qcr,qu,˙ qu)˙ qcr(t)+ ¯ e(˜qc+qcr,˙ ˜qc+˙ qcr,qu,˙ qu), and Iand 0are the identity and zeros matrices, respectively, with appropriate dimension.165 Remark 2. For design purposes of the controllers presented in this section, the uncontrolled DOF are assumed to be stabilized. 2.1. Nonlinear W2control To derive the nonlinear W2optimal control law, the following standard form is obtained from (5), considering no disturbances affecting the system (¯ δ(q,t)=0), P1:       ˙ x=f(x,qu,˙ qu,t)+¯ f(x,qu,˙ qu,t)+g(x,qu,t)τ, z=Rt 0˜ qc(t)dt,(6) in which zis the cost variable, chosen in order to ensure that all system dynamics are included in the cost functional.170 The nonlinear W2controller is designed in order to obtain a control law τ∈ U ⊆ Rcthat minimizes the cost functional J2=1 2||z(t)||2 W3,2,Y, =1 2Z∞ 0||z(t)||2 Y0+||˙ z(t)||2 Y1+||¨ z(t)||2 Y2+||... z(t)||2 Y3dt,(7) where ||(·)||2 Yi ,(·)0Yi(·), with Yi, for i∈ {0,1,2,3}, being a symmetric, positive definite tuning matrix that weights the influence of the states in the control objective. Hence, the optimal control problem (OCP) is stated as finding τ∗that leads to V2=min τ∈U 1 2||z(t)||2 W3,2,Y,s.t. P1.(8) The goal is to provide a smooth decay of the error between the generalized coordinates (as well as their time derivatives) and their references over time. Remark 3. In the nonlinear W2and W∞control approaches, the control inputs are not weighted directly in the optimizing175 index. The transient and steady-state performance are reached by tuning component-wise the influence of the cost variable and its time derivatives in to the cost functional. Therefore, aiming to solve (8), it is formulated via dynamic programming [27], from which the associated Hamiltonian is given by H2=∂0V2 ∂x˙ x+1 2z0Y0z+1 2˙ z0Y1˙ z+1 2¨ z0Y2¨ z+1 2 ... z0Y3 ... z,(9) with the boundary condition V2(0,t)=0. Expanding (9) yields H2=∂0V2 ∂xf(x,qu,˙ qu,t)+¯ f(x,qu,˙ qu,t)+g(x,qu,t)τ(10) +1 2x0 Y20 0 0Y10 0 0 Y0  x+1 2h˙ ˜ q0 c¯ C0Π¯ C˙ ˜ qc+2˙ ˜ q0 c¯ C0Π¯ d −2τ0¯ B0Π¯ C˙ ˜ qc−2τ0¯ B0Π¯ d+τ0¯ B0Π¯ Bτ+¯ d0Π¯ di, where Π,¯ M−1Y3¯ M−1. The HJB (Hamilton-Jacobi-Bellman) equation is obtained from (8) using the Hamiltonian (10), which in a compact form is given by ∂V2(x,t) ∂t+min τ∈U {H2(V2(x,t),x,qu,˙ qu,τ,t)}=0.(11) 3 The optimal control law τ∗can be obtained by minimizing the Hamiltonian (10) with respect to τ, which yields τ∗=¯ B−1¯ Mh−Y−1 30 0i∂V2 ∂x+¯ M−1¯ C˙ ˜ qc+¯ M−1¯ d.(12) This is indeed a min value of the optimization problem, since ∂2H∞/∂τ2=¯ B0Π¯ B>0. The HJ equation associated with the problem is obtained by replacing the optimal control law (12) in (10), and considering the time-varying nonlinear system (6), ∂V2(x,t) ∂t+H2(V2(x,t),x,qu,˙ qu,τ∗,t)=0.(13) A particular solution V2(x,t) of the HJ equation (13) is proposed in the following theorem.180 Theorem 1. Let V2be the parameterized scalar function V2(x,t)=1 2x0 Q K T K R S T S P  x>0,(14) such that the matrices Q,K,T,R,S, and Pare positive definite and verify Q−KR−1K>0and "Q K K R#−"T S#P−1hT Si>0, with T,Q,and Kobtained by solving the following Riccati equations −TY−1 3T+Y0=0,(15) −QY−1 3Q+2K+Y2=0,(16) −KY−1 3K+2QY−1 3T+Y1=0.(17) Besides, R,S, and P, are given by R=QY−1 3K−T,(18) S=QY−1 3T,(19) P=KY−1 3T.(20) Then, function V2(x,t)is a solution of the HJ equation (13). Proof 1. The proof is conducted by replacing (14) in (13) and using the properties of the inertia matrix. In the following the computation is performed by parts. Consider the HJ equation (13). Since, (14) is a time-invariant function, we have that ∂V2 ∂t=0, and (13) results in ∂0V2 ∂x˙ x+1 2x0 Y20 0 0Y10 0 0 Y0  x+1 2¨ ˜ q0 cY3¨ ˜ qc=0.(21) In addition, considering the candidate solution (14), the optimal control law (12) is given by τ∗=¯ B−1h−¯ MY−1 3Q−¯ MY−1 3K−¯ MY−1 3Tix+¯ C˙ ˜ qc+¯ d,(22) which leads to the following acceleration error dynamics ¨ ˜ qc=h−Y−1 3Q−Y−1 3K−Y−1 3Tix.(23) Accordingly, the closed-loop dynamics are given by ˙ x= −Y−1 3Q−Y−1 3K−Y−1 3T I0 0 0I0  x,(24) which is a closed-loop time-invariant system. Then, by replacing (24) in the first term of (21) yields ∂0V2 ∂x˙ x=x0 −QY−1 3Q+K−QY−1 3K+T−QY−1 3T −KY−1 3Q+R−KY−1 3K+S−KY−1 3T −TY−1 3Q+S−TY−1 3K+P−TY−1 3T  x.(25) Moreover, the last term of (21) is computed as 1 2¨ ˜ q0Y3¨ ˜ q=1 2x0 QY−1 3Q QY−1 3K QY−1 3T KY−1 3Q KY−1 3K KY−1 3T TY−1 3Q TY−1 3K TY−1 3T  x.(26) Thus, considering (25) and (26), the HJ equation (21) results in x0Nx =x0 N11 N12 N13 N21 N22 N23 N31 N32 N33  x=0,(27) with N11 ,−1 2QY−1 3Q+K+1 2Y2,N12 ,−1 2QY−1 3K+T,N13 , −1 2QY−1 3T,N21 ,−1 2KY−1 3Q+R,N22 ,−1 2KY−1 3K+S+1 2Y1, N23 ,−1 2KY−1 3T,N31 ,−1 2TY−1 3Q+S,N32 ,−1 2TY−1 3K+P, N33 ,−1 2TY−1 3T+1 2Y0.185 The matrix Nis decomposed by its symmetric and skewsymmetric components, resulting in 1 2x0(N+N0)x=x0 A11 A12 A13 ∗A22 A23 ∗ ∗ A33  x=0,(28) where the * terms indicate symmetry. One way to satisfy (28) is by solving the following set of algebraic Riccati equations A11 =−QY−1 3Q+2K+Y2=0,(29) A12 =−QY−1 3K+T+R=0,(30) A13 =−QY−1 3T+S=0,(31) A22 =−KY−1 3K+2S+Y1=0,(32) A23 =−KY−1 3T+P=0,(33) A33 =−TY−1 3T+Y0=0.(34) Firstly, Tis computed by solving (34). After, equations (18), (19) and (20) are obtained through (30),(31), and (33), respectively. Finally, Qand Kare computed through (29) and (32) by solving the following pair of cross-coupled Riccati equations −QY−1 3Q+2K+Y2=0,(35) −KY−1 3K+2QY−1 3T+Y1=0.(36) Thus, with the solutions of (29)-(34), we conclude the proof.  Furthermore, the stability of the closed-loop system (24) is verified by the following Theorem. Theorem 2. Let V2(x,t)>0be a solution of (13) given by the parameterized scalar function (14). Therefore, the closed-190 loop system, formed by the control law (12) and system (6), is asymptotically stable within the configuration domain Ω. Proof 2. From (13) and (14) we have that dV2 dt =−1 2x0 QY−1 3Q+Y2QY−1 3K QY−1 3T KY−1 3Q KY−1 3K+Y1KY−1 3T TY−1 3Q TY−1 3K TY−1 3T+Y0  x=−1 2x0Gx, (37) 4 where G=Λ1 Y−1 3Y−1 3Y−1 3 Y−1 3Y−1 3Y−1 3 Y−1 3Y−1 3Y−1 3  Λ1+ Y20 0 0Y10 0 0 Y0  ,(38) with Λ1,blkdiag(Q,K,T), and blkdiag(·)represents a block diagonal matrix whose diagonal elements are square matrices given in parentheses, and all off-diagonal blocks are zero matrices. Matrix (38) is composed of two terms, where the first is positive semi-definite and the second is positive definite, yielding G>0. Then, ˙ V2(x,t)<0, which ensures asymptotically stability for the closed-loop system in the Lyapunov sense. Accordingly, from (37) we have that Z∞ 0 dV2(x,t) dt dt =lim t→∞ V2(x(t),t)−V2(x(0),0),(39) =−Z∞ 0 1 2x0Gxdt =−1 2||z(t)||2 W3,2,Y=−c1, where x(0) is the value of the states xat the time instant t =0, and c1∈R+. Then, the W3,2,Yweighted Sobolev norm of the cost variable exists and is finite, consequently lim t→∞ x(t)=0.195 2.2. Nonlinear W∞control Consider again system (5), from which the following compact standard form is obtained P2:       ˙ x=f(x,qu,˙ qu,t)+¯ f(x,qu,˙ qu,t)+g(x,qu,t)τ+k(x,qu,t)¯ δ, z=Rt 0˜ qc(t)dt,(40) with zbeing the cost variable. The nonlinear W∞control in the weighted Sobolev space is posed in order to achieve the control law τ∈ U ⊆ Rcthat minimizes the following cost functional J∞=1 2||z||2 W3,2,Y−1 2γ2||¯ δ||2 L2,(41) for the worst case of the disturbances ¯ δ∈ L2[0,∞), considering a given sufficiently large W∞-index γ∈R+, where ||¯ δ||2 L2= R∞ 0¯ δ0¯ δdt. Definition 1. (Adapted from [28]) A generic function ¯ δ(t) :200 R+→Rc, for some c ∈N, belongs to the Lpspace (i.e. ¯ δ(t)∈ Lp[0,∞)) if and only if ||¯ δ(t)||Lp=R∞ 0Pk j=1|¯ δj(t)|pdt1 p<∞, where p ∈N∪ {∞} and ¯ δj(t)stands for the j-th element of the vector ¯ δ(t). Remark 4. It is assumed that the time derivatives of the dis-205 turbance vector are unknown, then, the L2-norm of the disturbance vector is considered into the cost functional (41). Therefore, the optimization problem is posed as V∞=min τ∈U max ¯ δ∈D n1 2||z||2 W3,2,Y−1 2γ2||¯ δ||2 L2o,s.t. P2,(42) with D=L2[0,∞). The goal is to provide trajectory tracking for the controlled DOF while being robust against external disturbances, unmodeled dynamics, and parametric uncertainties.210 The control law is obtained by solving a Min-Max optimization problem, which can be formulated via dynamic programming, using differential game theory. The HJBI (Hamilton- Jacobi-Bellman-Isaacs) equation associated to this problem is ∂V∞ ∂t+min τ∈U max ¯ δ∈D {H∞(V∞(x,t),qu,˙ qu,x,τ,¯ δ,t)}=0,(43) with the Hamiltonian H∞,∂0V∞ ∂x˙ x+1 2z0Y0z+˙ z0Y1˙ z+¨ z0Y2¨ z+... z0Y3 ... z−γ2¯ δ0¯ δ,(44) and boundary condition V∞(0,t)=0. The equation (44), in its expanded form, is written as H∞=∂0V∞ ∂xf(x,qu,˙ qu,t)+¯ f(x,qu,˙ qu,t)+g(x,qu,t)τ +k(x,qu,t)¯ δ+1 2x0 Y20 0 0Y10 0 0 Y0  x−1 2γ2¯ δ0¯ δ(45) +1 2h˙ ˜ q0 c¯ C0Π¯ C˙ ˜ qc+2˙ ˜ q0 c¯ C0Π¯ d−2τ0¯ B0Π¯ C˙ ˜ qc−2τ0¯ B0Π¯ d +τ0¯ B0Π¯ Bτ+¯ d0Π¯ d−2˙ ˜ q0 c¯ C0Π¯ δ−2¯ d0Π¯ δ+2τ0¯ B0Π¯ δ+¯ δ0Π¯ δi. The optimal control law, τ∗, and the worst case of the disturbances, ¯ δ∗, are obtained by computing the partial derivatives of (45) with respect to these variables and equaling them to zero as follows ∂H∞ ∂τ=g0∂V∞ ∂x−¯ B0Π¯ C˙ ˜ q−¯ B0Π¯ d+¯ B0Π¯ δ∗+¯ B0Π¯ Bτ∗=0,(46) ∂H∞ ∂¯ δ=k0∂V∞ ∂x−Π¯ C˙ ˜ q−Π¯ d+Π ¯ Bτ∗+Π¯ δ∗−γ2¯ δ∗=0.(47) Thus, considering (46), the optimal control law is given by τ∗=−¯ B0Π¯ B−1g0∂V∞ ∂x+¯ B¯ C˙ ˜ q+¯ B¯ d−¯ B¯ δ∗.(48) In addition, the worst case of the disturbances, ¯ δ∗, can be computed by premultiplying both sides of (47) by ¯ B0(q), and using (48), which yields ¯ δ∗=(¯ B0)−1 γ2k¯ B−g0∂V(x) ∂x.(49) Furthermore, through the second order partial derivatives of (45), which are given by ∂2H∞ ∂τ2=¯ B0Π¯ B>0,(50) ∂2H∞ ∂¯ δ2= Π −γ2I<0,(51) it can be verified that (48) and (49) are a Min-Max extremum of the optimization problem. Remark 5. The W∞-index γmust be selected appropriately such that the inequality (51) holds, ensuring feasibility to the optimization problem.215 The HJ PDE associated to this problem is obtained by replacing the optimal control law (48) and the worst case of the disturbances (49) in (43), yielding ∂V∞(x,t) ∂t+H∞(V∞(x,t),x,τ∗,¯ δ∗,t)=0.(52) Therefore, the OCP (42) results in finding a function V∞(x,t) that solves (52). 5 Remark 6. For the particular case of interest (see (5))k¯ B=g in (49), which leads the W2and W∞controllers to be equivalent. Therefore, the particular solution proposed in Theorem 1220 is also a solution to (52). The stability of the closed-loop system is stated in the following Theorem. Theorem 3. Let V∞(x,t)>0be a solution of (52) given by the parameterized scalar function (14). Therefore, the closed-loop225 system with the control law (48) and system (40) is asymptotically stable. Proof 3. From (52) and (14) we have that dV∞(x,t) dt =−1 2z0Y0z−1 2˙ z0Y1˙ z−1 2¨ z0Y2¨ z−1 2 ... z0Y3 ... z+1 2γ2¯ δ∗0¯ δ∗, =−1 2x0Gx +1 2γ2¯ δ∗0¯ δ∗,(53) where G>0is given by (38). In addition, from (5) in which k¯ B=g, the worst case of the disturbances is ¯ δ∗(t)=0 (see (49)). This implies that ˙ V∞(x,t)<0, ensuring asymp-230 totic stability for the closed-loop system in the Lyapunov sense. Consequently, similarly to (39), the W3,2,Yweighted Sobolev norm of the cost variable exists and is finite, which implies that limt→∞ x(t)=0. In the following we extend the definitions of Lq-stability and235 Lq-gain [28] and also of W-stability and W-gain [17], considering the weighted Sobolev norm. Definition 2. Suppose ¯ δ(t) : R+→Rn,z(t) : R+→Rc, for some n,c∈N, and z(t)=G(¯ δ(t)), such that G:Rn→Rc. The map Gis said to be Wm,p,σ-stable if there exist finite constants γ, v∈R+such that the inequality ||z(t)||2 Wm,p,σ≤γ2||¯ δ||2 Lp+v(54) holds for any ¯ δ∈ Lp[0,∞). Moreover, if γ=inf S in which S={γ∈R+:||z(t)||2 Wm,p,σ≤γ2||¯ δ||2 Lp+v,∀¯ δ∈ Lp[0,∞),v∈ R+}, then γis said to be the Wm,p,σ-gain of the system.240 Note that, ¯ δ∗is the worst case of the disturbances, thus it maximizes the Hamiltonian. Consequently, from (53), for a generic ¯ δ∈ L2[0,∞), we have that dV∞(x,t) dt ≤ − 1 2x0Gx +1 2γ2¯ δ0¯ δ.(55) Thus, by integrating both sides of (55) in time, we have that Z∞ 0 dV∞(x,t) dt dt ≤Z∞ 0 −1 2x0Gx +1 2γ2¯ δ0¯ δ!dt, lim t→∞ V∞(x(t),t)−V∞(x(0),0) ≤ − 1 2||z(t)||2 W3,2,Y+1 2γ2||¯ δ||2 L2,(56) which implies ||z(t)||2 W3,2,Y≤γ2||¯ δ||2 L2+2V∞(x(0),0). Then, from (54) and assuming a finite initial condition, the closed-loop system is W3,2,Y-stable. The related W3,2,Y-gain, i.e. the smallest γ∈R+, can be obtained from inequality (51). Remark 7. In contrast to the nonlinear H2and H∞control ap-245 proaches for mechanical systems [1, 10, 11, 13], the proposed W2and W∞controllers do not consider the control inputs directly in the cost functional, which relaxes the requirement that τ∈ L2[0,∞). In addition, this relaxation allows to formulate the control problem without any prior compensation of the250 gravity vector and the terms related to the desired trajectory. 3. Nonlinear W2and W∞control of entire underactuated mechanical systems This section formulates the nonlinear W2and W∞controllers for the class of underactuated mechanical systems with input coupling [25], by considering the entire dynamic model. Examples of mechanical systems with input coupling include the quadrotor UAV with tilted rotors [13], two-wheeled selfbalanced pendulum [24], autonomous helicopter [25], VTOL aircraft [25, 29], among others. The objective is to guide the controlled DOF throughout the desired trajectory, while the remaining ones are stabilized. Accordingly, equation (1) is partitioned in controlled and stabilized DOF as [13] "Mss(q)Msc(q) Mcs(q)Mcc(q)# | {z } M(q) "¨ qs ¨ qc# |{z} ¨ q +"Css(q,˙ q)Csc(q,˙ q) Ccs(q,˙ q)Ccc(q,˙ q)# | {z } C(q,˙ q) "˙ qs ˙ qc# |{z} ˙ q (57) +"gs(q) gc(q)# | {z } g(q) ="Bs(q) Bc(q)# | {z } B(q) τ+w(t), in which q=[q0 sq0 c]0, with qs∈Rn−ccorresponding to the stabilized DOF and qc∈Rcto the controlled ones. The controlled255 DOF must be selected so that the following assumptions hold. Assumption 2. The matrix Bc(q)is invertible within the configuration domain, i.e. rank(Bc(q)) =c, ∀q∈Ω, in which Ω⊆Rn. Assumption 3. The rows of Bs(q)are linearly independent260 within the configuration domain, i.e. rank(Bs(q)) =n−c= s,∀q∈Ω. Remark 8. Assumption 2 ensures that the image space generated by the matrix Bc(q)has dimension c, the same as qc, and allows to perform tracking of the controlled DOF. In addition,265 Assumption 3 ensures the image space generated by the matrix Bs(q)is not null, which provides actuation on the stabilized DOF by the control inputs. Physically, these assumptions ensure the system has input coupling, so the control inputs directly influence all system DOF.270 Aiming trajectory tracking, the error dynamics are written as M(qs,˜qc+qcr)"¨ qs ¨ ˜qc+¨ qcr#+C(qs,˜qc+qcr,˙ qs,˙ ˜qc+˙ qcr)"˙ qs ˙ ˜qc+˙ qcr# +g(qs,˜qc+qcr)=B(qs,˜qc+qcr)τ+w.(58) Hence, by defining the state vector x,˙ q0 s˙ ˜q0 c˜ qc0Rt 0˜ qc0dt0, equation (58) is represented in the state-space form as ˙ x= −M−1C0 0 [0I]0 0 0I0  x | {z } f(x,qs) + −M−1d 0 0  | {z } ¯ f(x,qs,t) + M−1 0 0  |{z} g(x,qs) u+ M−1 0 0  |{z} k(x,qs) w,(59) 6 where d,g(qs,˜qc+qcr)+M(qs,˜qc+qcr)"0 ¨ qcr#+C(qs,˜qc+qcr,˙ qs,˙ ˜qc+ ˙ qcr)"0 ˙ qcr#, and u=B(qs,˜qc+qcr)τis the generalized input vector. It is noteworthy that the vector dcomprises three terms, the first is related to the generalized forces induced on the mechanical system by gravity, the others are the generalized forces required275 to perform the desired trajectory. In order to design the W2and W∞controllers for underactuated mechanical systems with inputs coupling, we also assume that the desired trajectory is feasible and the forces and torques induced on the system by gravity and external disturbances can280 be handled using the control inputs. Accordingly, the following assumption is posed. Assumption 4. There exists qssuch that B⊥(qs,˜qc+ qcr)(w−d)=0, where B⊥(qs,˜qc+qcr)is a full rank left annihilator of B(qs,˜qc+qcr)(i.e B⊥B=0).285 Remark 9. Assumption 4 ensures that there exists the image space (or exact map using pseudo-inverse [30]) of vectors d and wfrom the generalized inputs uto the control inputs τ, for some configuration of the stabilized DOF. 3.1. Nonlinear W2control290 In order to derive the nonlinear W2optimal control law, system (59) is written in the following compact form P3:             ˙ x=f(x,qs,t)+¯ f(x,qs,t)+g(x,qs,t)u, zc=Rt 0˜ qc(t)dt, zs=˙ qs(t), (60) where zcand zsare the cost variables related to the controlled and stabilized DOF, respectively. They allow the appearance of both dynamics in the cost functional. Besides, it is assumed no disturbances affecting the system (w=0). The nonlinear W2controller is designed in order to obtain the control law u∈ U that minimizes the cost functional J2=1 2||zc||2 W3,2,Γ+1 2||zs||2 W1,2,Υ.(61) Therefore, the OCP is posed as finding u∗that leads to V2=min u∈U (1 2||zc||2 W3,2,Γ+1 2||zs||2 W1,2,Υ),s.t. P3,(62) in which Γ={Γ0, ..., Γ3}and Υ={Υ0,Υ1}, where Γiand Υj,295 for i∈ {0,1,2,3}and j∈ {0,1}, are symmetric, positive definite tuning matrices, that weight the influence of the states in the control objective. Remark 10. In the OCP (62), the time derivatives of the stabilized DOF are required to converge to zero, but there are no300 references for the stabilized DOF. This allows the controller to manipulate qsin order to achieve B⊥(qs,˜qc+qcr)(w−d)=0. Aiming to solve this problem, it can be formulated via dynamic programming, from which the HJB PDE is obtained ∂V2(x,t) ∂t+min u∈U {H2(V2(x,t),x,qs,u,t)}=0,(63) where the associated Hamiltonian is given by H2,∂0V2 ∂x˙ x+1 2z0 cΓ0zc+1 2˙ z0 cΓ1˙ zc+1 2¨ z0 cΓ2¨ zc+1 2 ... z0 cΓ3 ... zc +1 2z0 sΥ0zs+1 2˙ z0 sΥ1˙ zs, =∂0V2 ∂xf+¯ f+gu+1 2x0Dx +1 2¨ ˜ q0E¨ ˜ q,(64) with D,blkdiag(Υ0,Γ2,Γ1,Γ0),E,blkdiag(Υ1,Γ3), and the boundary condition V2(0,t)=0. Then, the optimal control law u∗, obtained by minimizing (64) with respect to u, is given by u∗=−Π−1g0∂V2 ∂x+C˙ ˜ q+d,(65) which is a min value of the optimization problem, since ∂2H2/∂u2=Π>0, with Π,M−1EM−1and ˙ ˜q=h˙ q0 s˙ ˜q0 ci0. The HJ equation associated to the problem, obtained by replacing the optimal control law (65) in (64), is written as ∂V2(x,t) ∂t+H2(V2(x,t),x,qs,u∗,t)=0.(66) Therefore, the OCP (62) results in solving the PDE (66) to obtain the solution V2(x,t). Then, similar to Theorem 1, a particular solution to (66) is proposed in the following theorem.305 Theorem 4. Let V2be the parametrized scalar function V2(x,t)=1 2x0 U0 0 0 0Q K T 0K R S 0T S P  x>0,(67) such that U,Q,K,T,R,S, and Pare positive definite matrices and verify Q−KR−1K>0and "Q K K R#−"T S#P−1hT Si>0, with U,Q,K, and Tobtained from the following Riccati equations −UΥ−1 1U+Υ0=0,(68) −TΓ−1 3T+Γ0=0,(69) −QΓ−1 3Q+2K+Γ2=0,(70) −KΓ−1 3K+2S+Γ1=0.(71) Besides, R,S, and Pare given by R=QΓ−1 3K−T,(72) S=QΓ−1 3T,(73) P=KΓ−1 3T.(74) Then, function V2(x,t)is a solution to the HJ equation (66). Proof 4. The proof is conducted similarly to the proof of Theorem 1, by replacing (67) in (66) and using the properties of inertia matrix.  The stability of the closed-loop system is demonstrated in the310 following Theorem. Theorem 5. Let V2(x,t)>0be a solution of (66) given by the parametrized scalar function (67). Therefore, the closed-loop system with the control law (65) and system (60) is asymptotically stable.315 7 Proof 5. From (66),(67), and D=blkdiag(Υ0,Γ2,Γ1,Γ0), we have that ˙ V2(x,t)=−1 2x0Fx, in which F,Λ2 Υ−1 1000 0Γ−1 3Γ−1 3Γ−1 3 0Γ−1 3Γ−1 3Γ−1 3 0Γ−1 3Γ−1 3Γ−1 3  Λ2+D,(75) with Λ2,blkdiag(U,Q,K,T). Matrix (75) is composed of two terms, where the first is positive semi-definite and the second is positive definite, yielding F>0. Then, ˙ V2(x,t)<0, which ensures asymptotically stability for the closed-loop system in the Lyapunov sense. Accordingly, the weighted Sobolev320 norms of the cost variables zcand zsexist, such that ||zc||2 W3,2,Γ+ 1 2||zs||2 W1,2,Υ<∞and, consequently, lim t→∞ x(t)=0. 3.2. Nonlinear W∞control Consider again system (59) to obtain the plant P4as follows P4:             ˙ x=f(x,qs,t)+¯ f(x,qs,t)+g(x,qs,t)u+k(x,qs,t)w, zc=Rt 0˜ qc(t)dt, zs=˙ qs(t). (76) The nonlinear W∞control of underactuated mechanical systems with input coupling is posed as finding the control law u∈ U that minimizes the cost functional, J∞=||zc||2 W3,2,Γ+1 2||zs||2 W1,2,Υ−1 2γ2||w||2 L2,(77) for the worst case of the disturbances w∈ L2[0,∞), considering a given sufficiently large γ∈R+. Hence, the OCP is given by V∞=min u∈U max w∈D (1 2||zc||2 W3,2,Γ+1 2||zs||2 W1,2,Υ−γ2||w||2 L2),s.t. P4.(78) The optimal control law is obtained by solving a Min-Max optimization problem, which is formulated again via dynamic programming using differential game theory. Therefore, the HJBI PDE is given by ∂V∞(x,t) ∂t+min u∈U max w∈D nH∞(V∞(x,t),x,qs,u,w,t)o=0,(79) where the associated Hamiltonian is defined as H∞,∂V∞(x,t) ∂x˙ x+1 2z0 cΓ0zc+1 2˙ z0 cΓ1˙ zc+1 2¨ z0 cΓ2¨ zc +1 2 ... z0 cΓ3 ... zc+1 2z0 sΥ0zs+1 2˙ z0 sΥ1˙ zs−1 2γ2w0w, =∂0V2 ∂xf+¯ f+gu +kw+1 2x0Dx +1 2¨ ˜ q0E¨ ˜ q−1 2γ2w0w(80) with the boundary condition V∞(0,t)=0. Similar to the procedure performed in Section 2.2, by optimizing the Hamiltonian (80) with respect to uand w, the optimal control law and the worst case of disturbance result in u∗=−Π−1g0∂V∞ ∂x+C˙ ˜ q+d−w∗,(81) w∗=1 γ2k0−g0∂V∞ ∂x.(82) which are a Min-Max extremum of the optimization problem,325 verified by ∂2H∞ ∂u2= Π >0 and ∂2H∞ ∂w2= Π −γ2I<0. Remark 11. Remark 5 holds similarly for ∂2H∞ ∂w2= Π −γ2I<0. The HJ PDE associated to the problem is obtained by replacing the optimal control law (81) and the worst case of disturbance (82) in (79), which in a compact form is written as ∂V∞(x,t) ∂t+H∞(V∞(x,t),x,qs,u∗,w∗,t)=0.(83) The OCP (78) results in solving the PDE (83) in order to find the solution V∞(x,t) . Remark 12. For the underactuated mechanical system with in-330 put coupling (see (57)), assuming u=B(qs,˜qc+qcr)τ, the error dynamics (59) possess k(x,qs,t)=g(x,qs,t), i.e. both matrices span the same column space, which means that any disturbance w∈ L2[0,∞)can be handled by an appropriate control input u∈ U. Accordingly, from (82) the W2and the335 W∞controllers become equivalent and the particular solution (67) proposed in Theorem 4 is also a solution to (83). The closed-loop stability is proven in the following Theorem. Theorem 6. Let V∞(x,t)>0be a solution of (83) given by (67). Therefore, the closed-loop system with the control law340 (81) and system (76) is asymptotically stable. Proof 6. The proof of this Theorem is similar to the proof of Theorem 5, since w∗(t)=0. In addition, for a generic w(t)∈ L2[0,∞) and a finite initial condition x(0), considering the control law (81) and the function V∞(x,t) given by (67), from (83) one can obtain Z∞ 0 dV∞(x,t) dt dt ≤Z∞ 0 −1 2x0Fx +1 2γ2w0w!dt,(84) lim t→∞ V∞(x(t),t)−V∞(x(0),0) ≤ − 1 2||zc||2 W3,2,Γ−1 2||zs||2 W1,2,Υ+1 2γ2||w||2 L2, and so ||zc||2 W3,2,Γ+||zs||2 W1,2,Υ≤γ2||w||2 L2+2V∞(x(0),0). Accordingly, the following inequalities are true ||zc||2 W3,2,Γ≤γ2||w||2 L2+2V∞(x(0),0) − ||zs||2 W1,2,Υ,(85) ||zs||2 W1,2,Υ≤γ2||w||2 L2+2V∞(x(0),0) − ||zc||2 W3,2,Γ.(86) Thus, from Theorem 6, we have that ||zc||2 W3,2,Γ+||zs||2 W1,2,Υ<∞, which implies ||zc||2 W3,2,Γ<∞and ||zs||2 W1,2,Υ<∞. Thereby,345 we can verify, from Definition 2 and through inequalities (85) and (86), that the closed-loop system is W3,2,Γ-stable from the disturbance to the output zc, and W1,2,Υ-stable from the disturbance to the output zs. In addition, the W3,2,Γ/W1,2,Υ-gain is the smallest γ∈R+that satisfies ∂2H∞ ∂w2= Π −γ2I<0.350 Since there are no references for the stabilized DOF, they are accommodated by the controller to physically provide disturbance rejection while the controlled DOF track the trajectory. The following proposition verifies the convergence of the stabilized DOF to a bounded stable condition.355 Proposition 1. Let qs(t) : R+→Rs, for some s ∈N. Suppose ||qs(0)|| <∞and Υ={Υ0,Υ1}, such that ||Υ0|| +||Υ1|| <∞. Therefore, if || ˙ qs(t)||2 W1,2,Υ<∞, then ||qs||L∞<∞. 8