Abstract:Infinite-horizon optimal control of constrained piecewise affine (PWA) systems has been approximately addressed by hybrid model predictive control (MPC), which, however, has computational limitations, both in offline design and online implementation. In this paper, we consider an alternative approach based on approximate dynamic programming (ADP), an important class of methods in reinforcement learning. We accommodate non-convex union-of-polyhedra state constraints and linear input constraints into ADP by designing PWA penalty functions. PWA function approximation is used, which allows for a mixed-integer encoding to implement ADP. The main advantage of the proposed ADP method is its online computational efficiency. Particularly, we propose two control policies, which lead to solving a smaller-scale mixed-integer linear program than conventional hybrid MPC, or a single convex quadratic program, depending on whether the policy is implicitly determined online or explicitly computed offline. We characterize the stability and safety properties of the closed-loop systems, as well as the sub-optimality of the proposed policies, by quantifying the approximation errors of value functions and policies. We also develop an offline mixed-integer linear programming-based method to certify the reliability of the proposed method. Simulation results on an inverted pendulum with elastic walls and on an adaptive cruise control problem validate the control performance in terms of constraint satisfaction and CPU time.

Intelligent Optimal Control of Constrained Nonlinear Systems Via Receding-Horizon Heuristic Dynamic Programming

Optimal Control for Hybrid Systems Based on Mixed Dynamic Programming

On Modeling and Optimal Control of a Class of Hybrid Systems

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Approximate Dynamic Programming for Constrained Piecewise Affine Systems with Stability and Safety Guarantees

A hybrid model-based optimal control method for nonlinear systems using simultaneous dynamic optimization strategies

Discrete‐Time Optimal Control of State‐Constrained Nonlinear Systems Using Approximate Dynamic Programming

A Digital Receding-Horizon Learning Controller for Nonlinear Continuous-time Systems

Optimal Control of Hybrid Systems Based on Dynamic Programming

Approximate dynamic programming for continuous state and control problems

Adaptive Dynamic Programming for Nonaffine Nonlinear Optimal Control Problem with State Constraints

Policy-Iteration-Based Finite-Horizon Approximate Dynamic Programming for Continuous-Time Nonlinear Optimal Control

Adaptive neural event‐triggered near‐optimal control for affined uncertain nonlinear discrete‐time system

Adaptive dynamic programming for optimal control of discrete‐time nonlinear system with state constraints based on control barrier function

A New Approach to Finite-Horizon Optimal Control for Discrete-Time Affine Nonlinear Systems via a Pseudolinear Method

Approximately Optimal Control of Discrete-Time Nonlinear Switched Systems Using Globalized Dual Heuristic Programming

H ∞ optimal control of unknown linear systems by adaptive dynamic programming with applications to time‐delay systems

Learning-Based N-Step Heuristic Dynamic Programming for Affine Nonlinear Optimal Regulation

Adaptive Multi-Step Evaluation Design With Stability Guarantee for Discrete-Time Optimal Learning Control

Robust adaptive dynamic programming for optimal nonlinear control design

Adaptive dynamic programming-based optimal control for nonlinear state constrained systems with input delay