Abstract:Infinite-horizon optimal control of constrained piecewise affine (PWA) systems has been approximately addressed by hybrid model predictive control (MPC), which, however, has computational limitations, both in offline design and online implementation. In this paper, we consider an alternative approach based on approximate dynamic programming (ADP), an important class of methods in reinforcement learning. We accommodate non-convex union-of-polyhedra state constraints and linear input constraints into ADP by designing PWA penalty functions. PWA function approximation is used, which allows for a mixed-integer encoding to implement ADP. The main advantage of the proposed ADP method is its online computational efficiency. Particularly, we propose two control policies, which lead to solving a smaller-scale mixed-integer linear program than conventional hybrid MPC, or a single convex quadratic program, depending on whether the policy is implicitly determined online or explicitly computed offline. We characterize the stability and safety properties of the closed-loop systems, as well as the sub-optimality of the proposed policies, by quantifying the approximation errors of value functions and policies. We also develop an offline mixed-integer linear programming-based method to certify the reliability of the proposed method. Simulation results on an inverted pendulum with elastic walls and on an adaptive cruise control problem validate the control performance in terms of constraint satisfaction and CPU time.

Online adaptation of controller parameters based on approximate dynamic programming

A Constructive Procedure Based on Dynamic Scaling for Adaptive Control of Nonlinearly Parameterized Nonlinear Systems

Parameter-dependent controller design for electro-hydraulic servo system

Automatic Parameter Adaptation for Quadrotor Trajectory Planning

Data-driven Direct Automatic Tuning Scheme for Fixed-Structure Digital Controllers of Hybrid Systems

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Online and model-free supplementary learning control based on approximate dynamic programming

Precision Motion Control of Hydraulic Actuators with Performance Guaranteed Online Parameter Estimation

Convergence Performance Oriented Data-Driven Tuning Method for Parameterised Controller Design with Cases Investigation

Generalized Parameter Estimation Method for Model-Based Real Time Optimization

Twin Deterministic Policy Gradient Adaptive Dynamic Programming for Optimal Control of Affine Nonlinear Discrete-time Systems

Logic Switching Based Online Periodic Adaptive Learning Control Algorithm Dealing with Unknown Period and Bound of the Uncertain Parameter

Parameter-Adaptive Approximate MPC: Tuning Neural-Network Controllers without Retraining

Online Adaptation of Kernel Learning Adaptive Predictive Controller

Autonomous Vehicle Control Through the Dynamics and Controller Learning

One Parameter Estimation-based Approximation-free Global Adaptive Control of Strict-feedback Nonlinear Systems

On-line and real-time controller tuning architecture based on Youla–Kucera controller parameterization

Exploring Optimal Controller Parameters for Complex Industrial Systems

Parameter tuning of modified adaptive backstepping controller for strict-feedback nonlinear systems

Incorporating approximate dynamic programming-based parameter tuning into PD-type virtual inertia control of DFIGs

Approximate Dynamic Programming for Constrained Piecewise Affine Systems with Stability and Safety Guarantees