Real-Time Implementation of Differentiable Predictive Control on Embedded Microcontroller Hardware: A Case Study

Real-Time Implementation of Differentiable Predictive Control on Embedded Microcontroller Hardware: A Case Study J´an Boldock´y∗Martin Gulan∗Draguna Vrabie∗∗ J´an Drgoˇna∗∗ ∗Slovak University of Technology, Faculty of Mechanical Engineering, N´amestie slobody 17, 812 31 Bratislava, Slovakia (e-mail: [email protected], [email protected]) ∗∗ Pacific Northwest National Laboratory, Richland, WA 99354, USA (e-mail: draguna.vr[email protected], jan.dr[email protected]) Abstract: This paper presents the embedded implementation of differentiable predictive control (DPC) in a real-time control application with fast dynamics. DPC is a model-based policy optimization method that utilizes automatic differentiation to learn explicit control policies. The laboratory device features an aeropendulum mechatronic system equipped with a low-cost 32-bit microcontroller development board. We aimed to explore the possibilities of utilizing the DPC with linear and nonlinear dynamics models on lean hardware in a setpoint tracking application while satisfying the input and state constraints. The resulting control performance, memory footprint, and execution time are evaluated for different lengths of the prediction horizon. Keywords: differentiable predictive control, embedded control, neural network, microcontroller 1. INTRODUCTION The importance of embedded control has grown tremendously over the recent decades, owing mainly to the reduction of size and cost of products from various industries, accelerated by the advancements in microprocessor technology. The usage of low-cost embedded computing units poses an additional challenge for control engineers because, besides the control problem itself, the task of working with units with limited memory and computational resources needs to be addressed as well. When dealing with various dynamical systems, employing model-based optimal control methods is often required. In particular, model predictive control (MPC) represents a well-established method with the ability to inherently handle constraints. However, this type of control requires solving the optimization problem, which is generally computationally demanding, making it prohibitive for realtime control on embedded computing hardware. In view of embedded MPC applications, one may resort to explicit MPC (EMPC) that removes the necessity to solve the optimization problem in real-time by precomputing its solution off-line using parametric programming, resulting in an explicit solution that maps the problem parameters to the decision variables (Bemporad, 2013). However, realtime implementation of EMPC solutions may still become intractable due to its exponential scalability. With the increasing popularity of deep learning, the socalled imitation learning method was introduced by Mor- ⋆J.B. and M.G. gratefully acknowledge the support under the grants APVV-18-002, APVV-22-0436, and 027STU-4-2024. This paper was funded by the European Union under Horizon Europe Grant Agreement no. 101079342 (Fostering Opportunities Towards Slovak Excellence in Advanced Control for Smart Industries). datch and Todorov (2014), which circumvents the curse of dimensionality of parametric programming solutions. In imitation learning, an MPC-based control policy is mimicked by a neural network (NN) (Reske et al., 2021; East et al., 2020). Inspired by trends of combining deep learning with MPC, Drgoˇna et al. (2022a) proposed the use of differentiable programming (Baydin et al., 2018) in order to obtain direct control policy gradients. This offline modelbased policy optimization method was named differential predictive control (DPC). Unlike imitation learning, DPC avoids the necessity of designing an MPC controller, making the controller synthesis more time-efficient. In this paper, we present a case study of the design and embedded implementation of a DPC policy in a reference tracking application on a benchmark mechatronic system. The DPC policies are computed using differentiable programming, implemented for linear, nonlinear, and neural dynamics models within the NeuroMANCER library (Drgoˇna et al., 2023). Subsequently, they are deployed on a 32-bit SAM3X8E Cortex-M3 microcontroller unit (MCU). Their performance is evaluated for various lengths of the prediction horizon. Besides a feasible hardware deployment, the experimental results obtained demonstrate that DPC can present a viable and efficient alternative to conventional MPC in embedded control applications. Please note that DPC is not able to provide stability or hard constraint guarantees inherently. In terms of that, the authors in (Mukherjee et al., 2022) and (Cortez et al., 2022; Xiao et al., 2023) propose two novel approaches that utilize neural Lyapunov and control barrier functions to address the problem of closed-loop stability and safety, respectively. In this case study, these concerns were not considered. 2. DIFFERENTIABLE PREDICTIVE CONTROL In this section, we briefly describe the theoretical background of differential predictive control. DPC is an offline model-based policy optimization algorithm that combines the principles of policy optimization from deep reinforcement learning with predictive and constraints handling capabilities of model predictive control. The control law is then represented by an arbitrary explicit function, most often a neural network, that maps the control parameters to decision variables. 2.1 Problem Formulation The DPC algorithm obtains the explicit solution of a parametric optimal control problem (OCP) formulated as follows: min Θ Eξk∼Pξh N−1 X k=0 ℓ(xk,uk,rk) + pN(xN)i(1a) s.t. xk+1 =f(xk,uk), k ∈NN−1 0,(1b) uk=πΘ(ξk),(1c) xk∈X,(1d) uk∈U,(1e) ξk∈Ξctrl ⊂Rn,(1f) where (1a) represents a sampled expectation of a parametrized MPC-like control objective function, e.g., reference tracking with terminal penalty pN. The system dynamics model (1b) is used to predict N-steps ahead future output trajectories, given the initial condition x0and the control actions generated by the control policy (1c). Assuming the differentiability of the system model (1b), the problem can be formulated with any popular model representation, e.g., linear-state space model (LSSM), nonlinear state space model (NL-SSM), neural state space model (NSSM), ordinary differential equations (ODEs), or neural ODEs. Furthermore, due to its sampling-based nature, the DPC algorithm can be extended to stochastic cases, including parametric and additive uncertainties in the system dynamics (Drgoˇna et al., 2022b). In (1d) and (1e), the formulation captures sets of state Xand input constraints U, respectively. The problem parameters ξkthat include initial conditions are sampled from a known probability distribution Pξ. Besides initial conditions, the OCP parameters can include desired reference trajectories, or constraints parametrizations. In this paper, we consider the current state of the system xkand future reference trajectory R= [rT k,...,rT k+N]T. Thus, the control parameters vector is ξk= [xT kRT]T. In Drgoˇna et al. (2022a), the authors also include lower and upper bounds for future output trajectories, which are redundant in our case, considering that we are working with constant constraints throughout the entire control problem. The neural control policy πΘ(·) can be, in principle, parameterized by arbitrary differentiable function. For convenience and expressive power in this work, we have chosen a deep NN with fully connected layers. The neural control policy and system model compose a closed-loop dynamical system, through which we can sample the control parameters. This allows us to compute the direct policy gradients with respect to the desired control objective. 2.2 DPC Policy Gradients As introduced by Drgoˇna et al. (2022a) and later expanded by Cortez et al. (2022) and Mukherjee et al. (2022), the fundamental innovation of the DPC technique lies in implementing the parametric OCP in a differentiable programming language, such as PyTorch, Jax, or Julia. Note that, in order to be able to solve the OCP (1) using gradient-based optimization, the Lagrangian methodology is applied using the penalty method. This comprises using the constraints (1d) and (1e) as penalties pxand pu in the objective function, which results in the following mathematical representation: LDPC = min Θ Eξk∼Pξh N−1 X k=0 Qr||ri k−xi k||2 2+Qu||ui k||2 2 +Qcxpx(xk) + Qcupu(uk) + QcNpN(xN)i(2a) s.t. xk+1 =f(xk,uk), k ∈NN−1 0,(2b) where the first term on the right-hand side represents the reference tracking objective with the corresponding weighting matrix Qr. The second term represents the control action weighted by Qu. Matrices Qcxand Qcu include weights for upper [x,u] and lower [x,u] state and input inequality penalty constraints. This allows us to represent the problem as a computational graph, which can be perceived as an MPC-inspired physics-informed neural architecture (Nghiem et al., 2023). The differentiability of the implemented program (1) in the form of computational graphs allows us to specifically compute its sensitivities (gradients) via automatic differentiation. More concretely, this allows us to employ the chain rule to compute the policy gradients in a scalable way given as follows: ∇ΘℓDPC =∂ℓ(x,u,r) ∂Θ+∂px(x) ∂Θ+∂pu(u) ∂Θ+∂pN(xN) ∂Θ =∂ℓ(x,u,r) ∂x ∂x ∂u ∂u ∂Θ+∂ℓ(x,u,r) ∂u ∂u ∂Θ +∂px(x) ∂x ∂x ∂u ∂u ∂Θ+∂pu(u) ∂u ∂u ∂Θ +∂pN(xN) ∂x ∂x ∂u ∂u ∂Θ,(3) where ∂u/∂Θrepresents the policy gradients computed using backpropagation through time algorithm (BPTT) (Puskorius and Feldkamp, 1994). Now, these gradients can be effectively used to train the explicit neural control policy πΘ(·) to minimize the loss function ℓDPC, composed of the control objective ℓ(x,u,r), terminal penalty pN(xN), and state and input constraints penalty functions px(x) and pu(u), respectively. For more details on the policy optimization algorithm in different settings, we refer the reader to Drgoˇna et al. (2022a,b); Cortez et al. (2022); Mukherjee et al. (2022); Drgoˇna et al. (2024). 3. CASE STUDY – AEROPENDULUM CONTROL In this section, we describe a procedure for obtaining and implementing DPC on embedded microcontroller hardware in the task of setpoint tracking 1, illustrated in Fig. 1 1The code to the case-study is available at github.com/xboldocky. m βl b K Open-loop data from experiment Linearization and discretization Control design Export to C code Microcontroller implementation initial conditions references synthetic dataset Ax(k) + Bu(k) control policy system model control objective constraint penalties reference tracking simulation feedback System identification u(k) y(k) kk y(k) u(k)y(k) u(k) Fig. 1. A simplified block diagram of the design and implementation of DPC on embedded microcontroller hardware. 3.1 System description As a testbed device, we opted for AeroShield introduced recently in (Vargov´a et al., 2023), which was developed in the scope of the AutomationShield initiative. In a nutshell, this miniature experimental device features a DC motor with a shaft mounted propeller that generates a thrust force causing the pendulum arm to lift. It is a single-input single-output system, since the only manipulated input is the voltage applied to the motor, and the only measured output is the angular displacement of the pendulum arm. In addition, the device comes with a C/C++ application programming interface, which was used in this study. 3.2 System identification The mathematical dynamics model will play a crucial role in controller design since it presents only information about system dynamics for obtaining the control policy. Nevertheless, our case study features grey-box system identification using the first-principle model represented by a set of nonlinear ODEs. We remark that a similar procedure for system identification was employed in (Vargov´a et al., 2023). The first-principle model of the system is described by the following ODE: ml2¨ β(t) = lT(t)−mgl sin β(t)−c˙ β(t),(4) where land mdenote the length and mass of the pendulum, respectively, gdenotes the gravitational acceleration, bis the coefficient of viscous damping in the rotary bearing joint, and βdenotes the angular displacement of the pendulum arm. Note that the center of mass in (4) was considered to be at the end of the pendulum arm. This second-order differential equation can be transformed into a state-space representation as follows: ˙x1(t) = x2(t),(5a) ˙x2(t) = K mlu2(t)−g lsin x1(t)−c ml2x2(t),(5b) where x1(t) = β(t) and x2(t) = ˙ β(t). The lift force T(t) in correlation with the voltage input u(t) is assumed to be quadratic with respect to the thrust coefficient K. The experimental data for system identification was acquired as an open-loop response to the amplitudemodulated pseudo-random binary input signal in the range [1.1,2.3] V. Two experiments were carried out, each lasting 48 s, where the first was used for system identification and the second for model validation. The system identification of the nonlinear ODE model was performed using the System Identification Toolbox provided by MATLAB. The identification process was performed using the nonlinear least-squares solver with Euler’s integration method. Input Table 1. Identified parameter values of the first-principle nonlinear grey-box model. Parameter Initial guess Identified value Gravitational acceleration (g) 9.81 m s−29.81 m s−2 Damping coefficient (c) 0.001 N m s−15.82 ×10−4Nms−1 Length of pendulum arm (l) 0.1 m 0.1170 m Mass (m) 0.01 kg 0.0327 kg Gain of the fan (K) 0.05 N V−10.05 N V−1 inter-sample behavior was set to zero-order hold. In Tab. 1, the values of identified model parameters are given along with the initial guess of parameters, of which gravitational acceleration was the only known parameter. The identified nonlinear ODE model (5) was then linearized around the operating point of π/6and discretized assuming the sampling time of 10 ms and zero-order hold method, resulting in the following linear-state space model: x(k+ 1) = 0.9964 0.0099 −0.7204 0.9835x(k) + 0.0021 0.4285u(k).(6) As evidenced in Fig. 2, the nonlinear first-principle and linearized models match the experimental validation data with a 84 % and 55 % fit, respectively. Additionally, system identification, using the neural blackbox model, was attempted primarily to demonstrate the capability of DPC in a setting when the first principle model is not or is only partially evident. Initially, we opted to identify the system using a neural ODE. Although this attempt yielded a model with sufficient goodness-of-fit, the model was inconsistent in capturing the state transition, i.e., the correlation in (5a) was not persistent even with the use of the physics-informed loss within the training. This problem was solved with a universal differential equation (UDE), which allows us to combine first-principle information with machine learning (Rackauckas et al., 2020). We thus considered the following model structure: ˙x1(t) = x2(t),(7a) ˙x2(t) = fu(u2(t)) −fx(x2(t)) −g lsin x1(t),(7b) where fuand fxare neural networks with 3 fully connected hidden layers with 30 neurons per layer. In (7b), the last term is extended from (5b), which benefited the identification process by implying the oscillatory behavior of the model. The parameters gand lwere fixed and equal to the identified values of the first-principle model, see Tab. 1. The system identification using the UDE model was implemented in the NeuroMANCER framework, where the Adam optimizer was used with full-batch gradient descent and a learning rate of 0.003. This regression setting was executed by fitting the 100 step ahead model rollouts to the corresponding sections of experimental data using 15 20 25 30 35 40 Time (s) 0 0.5 1 1.5 Angle (rad) Experiment Nonlinear model response 84.26% UDE model response 77.09% Linear model response 55.93% Fig. 2. Simulated response comparison of identified models with experimental data. mean square error loss. The model was simulated using the fourth order Runge-Kutta method with a step size of 0.01. The system identification process finished in 200 epochs, yielding a model with response of 77 % fit; see Fig. 2. Note that all the considered models were further enhanced with an integrator state x3to eliminate steady-state error in setpoint tracking. The integrator state is defined as follows: ˙x3(t) = e(t) = x1(t)−r(t). 3.3 Controller design When designing the DPC control policy, the first step is to generate a synthetic dataset of control parameters ξk. The dataset values are randomly generated within the range of the system's operating domain, assuming a uniform probability distribution. Within the training process, the nature of the DPC is adjusted according to its control objective and constraints. In our case, the loss function is represented in (2a), where both state and input constraints are constant throughout the case study and will be further defined by scalar values. For training the control policies based on the linearized model (6), the weighing matrices were manually finetuned as Qr= diag(5,0.5,0.05) and Qu= 0.01. The first value on the diagonal of Qrcorresponds to the control error of the angular displacement, the second value to the angular velocity, and the third to the summation of the control error with respect to zero. Since the purpose of the integrator state is to eliminate the steady-state error with respect to the reference, the integrator value is shifted by the current value of the reference in the implementation. State constraints are weighted by Qcx=0.1 for both upper and lower bound, while input constraints are weighted by Qcu= 0.01. The terminal penalty term weighted by QxN= 5 was considered only in the case of DPC for N=12. The constraints values for the first two states were x= [−1,−8]Tand x= [2.5,8.0]T. Constraints were not imposed on the integrator state x3. The input constraints values represent the physical limits of the hardware, i.e., the voltage limit of the DC motor where the lower bound is u=0.0 and upper u=3.7. Initially, the parametric OCP for DPC policies, based on nonlinear ODE and UDE models, was set up equally for the prediction horizon of 6 and 12. The state weight matrix was tuned to Qx= diag(10,0.1,6), input weight to Qu= 0.01, and input and state constraint penalties to Qcu= 1 and Qcx= [1.0,1.0]T, respectively. Terminal penalty was QN=7. Input constraints preserved the values of physical hardware limitations, while the state constraints were narrowed to x=[−0.5,−8.0]Tand x=[1.5,3.0]T, different to the setup with the linear model. However, within the initial control simulations and experiments, it was evident that the controller based on the UDE model with N= 6 had a significant tendency to undershoot, thus the state lower bound values were changed to x=[−0.5,−3]T. Also, the terminal penalty weight QxNwas reduced for both ODE and UDE based controllers to 5 and 3, respectively, primarily to improve the convergence within the training process. Lastly, these controllers were improved by adding the control smoothing term PN k=1 Qdu||ui k+1 −ui k||2 2to the loss function, where the input smoothing weight Qdu was set to 10 and 5 for policies with prediction horizons of N= 12 and N= 6, respectively. Within the training process, these nonlinear continuous models were simulated using the fourth order Runge-Kutta method with the step of 0.01, which represents the sample time of the controllers. When choosing the hyperparameters, it is necessary to choose a NN architecture adequate to the capabilities of the given control hardware and sampling frequency. With that in mind, for policies based on the linear model, the classical ReLU activation function was used. However, after some analysis, it was evident that due to its onesided rectification, it was mostly unable to satisfy the upper input constraint. The alternative was to use ReLU6, where the upper bound is also present. The decision to use a piecewise affine activation function for DPC learned on a linear model is supported by the fact that the optimal solution can be represented by a piecewise affine function for such an OCP. Moreover, for policies based on nonlinear models, we opted for the sigmoid activation function. Lastly, the AdamW optimizer was used, with a decoupled weight decay regularization of a decay coefficient 0.01 and a full batch gradient descent. The synthetic dataset of 9000 data samples was divided into thirds for training, testing, and validation. For the case of the linear model, the learning rate amounted to 0.001, while for nonlinear models, the learning rate was reduced to 1 ×10−6. 3.4 Embedded hardware deployment A nontrivial part of hardware deployment is the integration of the feedforward function, where code optimization plays a pivotal role in effective implementation. The programmer is left to decide whether to store the parameters of the NN in the volatile or the non-volatile memory. The former is of lesser capacity; however, the evaluation is faster. On the other hand, if a high sampling rate is not required, the latter can be used, allowing a bigger network to be stored. In this study, a custom application programming interface was used for the evaluation of fully connected deep neural networks. 4. RESULTS AND DISCUSSION In this section we will present and discuss the results of the embedded implementation of DPC on the AeroShield. The DPC policies were constructed for different lengths of the prediction horizon and a constant sampling time of Table 2. Hardware requirements and control performance indicators of the controllers based on linear model. Control Memory footprint Execution time [ms] RMSEβ Complexity of controller 1 policy N[B] mean max [rad] L×nnodes nregions 3 7302 2.34 2.50 0.0719 3×17 – DPC 6 9328 3.45 3.73 0.067 3×20 – 12 14 656 6.07 6.17 0.0355 4×20 – 3 16 160 0.19 1.93 0.0601 – 93 EMPC 6 61 008 0.59 2.06 0.0548 – 279 12 672 784 † † † – 2180 1L,nnodes,nregions – number of layers, nodes per layer, and regions. †Exceeds the available flash memory of 512 kB. 10 ms. Note that this paper is not focused on comparing DPC with EMPC, but rather on demonstrating the control performance of DPC alongside an optimal one. In all cases, the loss function and weighting matrices of the DPC and EMPC controllers were identical, varying only for the DPC with N= 12, where a terminal penalty with QxN= 5 was added. That resulted in a slight reduction of the overshoot. In the case of the prediction horizon N= 3, DPC exhibited a slightly worse control performance, judging only by the root-mean-square error (RMSE) for the pendulum angle reported in Tab. 2. When increasing the prediction horizon of the EMPC to N= 6 steps, its size in terms of the number of regions increased from 93 to 778. To make it deployable on the MCU, it was necessary to reduce its size. Our choice was not to consider the state constraints, reducing the number of regions to 279. In Fig. 3, one may observe the superior performance of the optimal controller, whereas the suboptimality of DPC resulted in a slower and more oscillatory control performance, presumably caused by the approximation errors. Next, when we doubled the prediction horizon length to N= 12 steps, it became evident that the EMPC solution is no longer deployable on the MCU, primarily due to the shortage of available non-volatile memory required to store its 1342 regions. On the other hand, the embedded implementation of an equivalently tuned and constrained DPC controller was still feasible. However, it was necessary to increase the size of the NN to 4 hidden layers with 20 nodes per layer. In Fig. 3, one may see that the resulting control performance is superior to the previous ones, mainly owing to the longer time window of future reference preview. This fact is also supported by the RMSE in Tab. 2. Note also that with increasing complexity, unlike the memory-intensive EMPC, the embedded DPC implementation is likely to encounter hardware limits due to prohibitive evaluation time rather than memory requirements. Although we have demonstrated that DPC can outperform EMPC, it should be noted that in the case when the system-model mismatch is significantly high, in other words, when integral action is necessary to provide offsetfree control, DPC may fail. This occurs when the integrator exceeds the range assumed within the training process. The controller then outputs inadequate control action. Differentiable control policies, based on nonlinear UDE and ODE models, were tested in a reference tracking control experiment, where the reference values extended beyond the linearization point, where the values ranged from 0.35 to 1.2 rad as shown in Fig. 4. All controllers were represented by a NN of the same size, with 3 hidden layers of 20, 20, and 10 neurons ascendingly, implying that the execution time was to a great extent the same for all controllers, more specifically 6.46 ms, making the implementation feasible in real time considering the sampling period of 10 ms. It is worth mentioning that the NN of equal size with the Tanh activation function would have taken 7.71 ms on average for a single forward pass, which made it unfeasible for real-time implementation when taking into account the auxiliary computation. Nevertheless, according to the RMSE metric, the policy based on the ODE model of N=12 was superior among the four controllers, with a value of 0.0772, followed by the policy based on the UDE model of N=12 with an RMSE value of 0.0925. The control policies of N=6 followed the same pattern, where the ODE based controller performed better with RMSE of 0.1031 compared to its UDE based counterpart with RMSE of 0.01115. These results coincide considering that the ODE model captured system dynamics with greater precision compared to the UDE model; see Fig. 2. In Fig. 4, one may observe that UDE based policies had a greater tendency to overshoot in contrast to the more conservative behavior of ODE based models. Also, the UDE based controller with N= 6 persisted with undershoot, despite having a more rigorous lower bound constraint of x2. As per Fig. 3 and 4, we may observe that constraint violation did not occur at any point of the control experiments. 5. CONCLUSION In this paper, we have experimentally demonstrated that differentiable predictive control can be viable and efficient in embedded control applications for linear and nonlinear models. Within the case study of controlling a nonlinear mechatronic system with fast dynamics we concluded that with an increasing prediction horizon, DPC could yield a better performance than the optimal EMPC in terms of real-time tracking performance. In addition, we explore the capabilities of DPC in the realm of nonlinear and neural models, and it transpires that DPC is an effective method for all types of models featured in the study. Our future research will focus primarily on accelerating the embedded evaluation of the neural control policy during runtime, which could be addressed, for example, via C code optimization or by using different NN architectures, such as fast feedforward networks. In addition, eliminating the use of integrator state while achieving offset-free performance may also be considered a motivation for future work. REFERENCES Baydin, A.G., Pearlmutter, B.A., Radul, A.A., and Siskind, J.M. (2018). Automatic differentiation in machine learning: a survey. Journal of Machine Learning Research, 18(153), 1–43. URL http://jmlr.org/ papers/v18/17-468.html. Bemporad, A. (2013). Explicit model predictive control. In J. Baillieul and T. Samad (eds.), Encyclopedia of Systems and Control, 1–9. Springer London. Cortez, W.S., Drgoˇna, J., Tuor, A., Halappanavar, M., and Vrabie, D. (2022). Differentiable predictive control with safety guarantees: A control barrier function approach. In 61st Conference on Decision and Control, 932–938. 0.35 0.55 0.75 Angle (rad) 10 15 20 25 30 35 40 45 50 Time (s) 0 1 2 3 3.7 Voltage (V) Fig. 3. Performance of LSSM-based DPC and EMPC in a reference tracking experiment near the operating point. Note: reference (solid black), constraints (dashed black), EMPC (blue), DPCN=6 (red), DPCN=12 (green). -0.5 0 0.5 1 1.5 Angle (rad) 0 5 10 15 20 25 30 35 40 45 Time (s) 0 1 2 3 3.7 Voltage (V) Fig. 4. Performance of DPC based on nonlinear models in a reference tracking experiment. Note: reference (solid black), constraints (dashed black), DPCODE N=6 (blue), DPCODE N=12 (green), DPCUDE N=6 (pink), DPCUDE N=6 (red). Drgoˇna, J., Tuor, A., Koch, J., Shapiro, M., and Vrabie, D. (2023). NeuroMANCER: Neural Modules with Adaptive Nonlinear Constraints and Efficient Regularizations. URL https://github.com/pnnl/neuromancer. Drgoˇna, J., Kiˇs, K., Tuor, A., Vrabie, D., and Klauˇco, M. (2022a). Differentiable predictive control: Deep learning alternative to explicit model predictive control for unknown nonlinear systems. Journal of Process Control, 116, 80–92. Drgoˇna, J., Mukherjee, S., Tuor, A., Halappanavar, M., and Vrabie, D. (2022b). Learning stochastic parametric diferentiable predictive control policies. IFACPapersOnLine, 55(25), 121–126. 10th IFAC Symposium on Robust Control Design. Drgoˇna, J., Tuor, A., and Vrabie, D. (2024). Learning constrained parametric differentiable predictive control policies with guarantees. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 1–12. East, S., Gallieri, M., Masci, J., Koutnik, J., and Cannon, M. (2020). Infinite-horizon differentiable model predictive control. arXiv: https://arxiv.org/abs/2001.02244. Mordatch, I. and Todorov, E. (2014). Combining the benefits of function approximation and trajectory optimization. In Robotics: Science and Systems, volume 4, 23. Mukherjee, S., Drgoˇna, J., Tuor, A., Halappanavar, M., and Vrabie, D. (2022). Neural Lyapunov differentiable predictive control. In 61st Conference on Decision and Control, 2097–2104. Nghiem, T.X., Drgoˇna, J., Jones, C., Nagy, Z., Schwan, R., Dey, B., Chakrabarty, A., Di Cairano, S., Paulson, J.A., Carron, A., Zeilinger, M.N., Shaw Cortez, W., and Vrabie, D.L. (2023). Physics-informed machine learning for modeling and control of dynamical systems. In 2023 American Control Conference, 3735–3750. Puskorius, G. and Feldkamp, L. (1994). Truncated backpropagation through time and Kalman filter training for neurocontrol. In 1994 International Conference on Neural Networks, volume 4, 2488–2493. Rackauckas, C., Ma, Y., Martensen, J., Warner, C., Zubov, K., Supekar, R., Skinner, D., Ramadhan, A., and Edelman, A. (2020). Universal differential equations for scientific machine learning. arXiv: https://arxiv.org/abs/2001.04385. Reske, A., Carius, J., Ma, Y., Farshidian, F., and Hutter, M. (2021). Imitation learning from MPC for quadrupedal multi-gait control. In International Conference on Robotics and Automation, 5014–5020. Vargov´a, A., Boldock´y, J., Gulan, M., Tibensk´y, P., Mikul´aˇs, E., and Tak´acs, G. (2023). Aeroshield: An opensource propeller-driven pendulum device for control engineering education. IFAC-PapersOnLine, 56(2), 9600– 9605. 22nd IFAC World Congress. Xiao, W., Wang, T.H., Hasani, R., Chahine, M., Amini, A., Li, X., and Rus, D. (2023). Barriernet: Differentiable control barrier functions for learning of safe robot control. IEEE Transactions on Robotics, 39(3), 2289–2307.