Abstract:The application of supervised learning techniques in combination with model predictive control (MPC) has recently generated significant interest, particularly in the area of approximate explicit MPC, where function approximators like deep neural networks are used to learn the MPC policy via optimal state-action pairs generated offline. While the aim of approximate explicit MPC is to closely replicate the MPC policy, substituting online optimization with a trained neural network, the performance guarantees that come with solving the online optimization problem are typically lost. This paper considers an alternative strategy, where supervised learning is used to learn the optimal value function offline instead of learning the optimal policy. This can then be used as the cost-to-go function in a myopic MPC with a very short prediction horizon, such that the online computation burden reduces significantly without affecting the controller performance. This approach differs from existing work on value function approximations in the sense that it learns the cost-to-go function by using offline-collected state-value pairs, rather than closed-loop performance data. The cost of generating the state-value pairs used for training is addressed using a sensitivity-based data augmentation scheme.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: in model predictive control (MPC), how to construct myopic MPC strategies through supervised learning methods to reduce online computation without sacrificing controller performance. Specifically, the paper proposes a new method, that is, to derive the cost - to - go function through offline supervised learning instead of directly learning the MPC strategy. This value function serves as an estimate of the cost to the end point in myopic MPC, significantly reducing the online computation requirements while maintaining the controller performance. ### Main problems and solutions 1. **Problem background** - **Computational burden of MPC**: Traditional MPC needs to solve optimization problems online, which is a challenge for embedded systems because these systems have limited resources and require fast sampling times. - **Limitations of existing methods**: Existing methods such as explicit MPC can pre - calculate parameterized strategies, but often sacrifice the performance guarantees brought by online optimization. 2. **Proposed solutions** - **Offline supervised learning**: Derive the cost - to - go function through offline supervised learning instead of directly learning the MPC strategy. - **Introduce descent property constraint**: Add a descent property constraint during the learning process to ensure that the cost of subsequent states is lower than that of the current state, thereby improving the effectiveness and stability of learning. - **Train with optimal state - action - value tuples**: Different from existing methods, this method uses offline - generated state - action - value tuples for training instead of closed - loop performance data. ### Key contributions - **Novel learning method**: Approximate the value function through supervised learning combined with the descent property constraint. This method not only improves the closed - loop performance but also can handle real - time changes in controller parameters. - **Experimental verification**: Through a benchmark continuous stirred - tank reactor (CSTR) example, it is shown that a good fit based solely on the mean - squared - error (MSE) loss does not always lead to good closed - loop performance, and the learning method with the descent property constraint can significantly improve the closed - loop performance. ### Formula summary The key formulas involved in the paper are as follows: - **MPC problem**: \[ V^*(x(t))=\min_{\mathbf{x}_k,\mathbf{u}_k}\sum_{k = 0}^{N - 1}\ell(\mathbf{x}_k,\mathbf{u}_k)+\ell_N(\mathbf{x}_N) \] Subject to: \[ \begin{aligned} &\mathbf{x}_{k + 1}=f(\mathbf{x}_k,\mathbf{u}_k)\quad\forall k\in I_0:N - 1\\ &\mathbf{x}_k\in X,\mathbf{u}_k\in U\\ &\mathbf{x}_0=x(t),\mathbf{x}_N\in X_f \end{aligned} \] - **Myopic MPC strategy**: \[ \min_{\mathbf{u}\in U}\ell(x(t),\mathbf{u})+V_f(\mathbf{x}^+) \] Subject to: \[ \mathbf{x}^+=f(x(t),\mathbf{u})\in X \] - **Learning objective function**: \[ \hat{\theta}_2=\arg\min_{\theta}\frac{1}{n_{tr}}\sum_{i = 1}^{n_{tr}}\|V(\mathbf{x}_i;\theta)-V^*(\mathbf{x}_i)\|^2 \] With additional descent property constraint: \[ V(

On Building Myopic MPC Policies using Supervised Learning

SOMTP: A Self-Supervised Learning-Based Optimizer for MPC-Based Safe Trajectory Planning Problems in Robotics

Safe and Near-Optimal Policy Learning for Model Predictive Control using Primal-Dual Neural Networks

Neural Lyapunov Model Predictive Control: Learning Safe Global Controllers from Sub-optimal Examples

GAN-MPC: Training Model Predictive Controllers with Parameterized Cost Functions using Demonstrations from Non-identical Experts

Learning the cost-to-go for mixed-integer nonlinear model predictive control

A Sensitivity-based Data Augmentation Framework for Model Predictive Control Policy Approximation

Fast Nonlinear Model Predictive Control Combining Online Trajectory Optimization and Value Function Regression

Learning a convex cost-to-go for single step model predictive control

Safe and Stable Closed-Loop Learning for Neural-Network-Supported Model Predictive Control

MPC with Sensor-Based Online Cost Adaptation

MPC of Uncertain Nonlinear Systems with Meta-Learning for Fast Adaptation of Neural Predictive Models

Blending MPC & Value Function Approximation for Efficient Reinforcement Learning

Stability-informed Bayesian Optimization for MPC Cost Function Learning

Constraints-Informed Neural-Laguerre Approximation of Nonlinear MPC with Application in Power Electronics

Regret Analysis of Learning-Based MPC With Partially-Unknown Cost Function

Model Predictive Control via On-Policy Imitation Learning

Neural Networks for Fast Optimisation in Model Predictive Control: A Review

Learning Sampling Distributions for Model Predictive Control

Faster Model Predictive Control via Self-Supervised Initialization Learning

Approximate non-linear model predictive control with safety-augmented neural networks