Abstract:In this paper, a novel adaptive dynamic programming (ADP)-based optimal control method is developed for discrete-time systems subject to constraints and disturbances. Particularly, a safe policy iteration scheme is designed to handle state and input constraints, including both hard and soft constraints, by converting the original policy improvement strategy into a constrained optimization problem with a prescribed state cost function. After that, an actor-critic-disturbance framework is introduced to address the constrained optimal control problem. The robust safety against disturbances is treated as a two-player zero-sum game, where the actor and disturbance neural networks are used to approximate the optimal control input and the disturbance policy, respectively. The convergence property of the proposed algorithm is analyzed, and the multi-step version of the proposed ADP scheme is derived based on this property. Simulation results are demonstrated and discussed to validate the effectiveness and performance of the proposed method. Note to Practitioners—Addressing constraints in optimal control problems is essential for guaranteeing the safe operation of controlled systems. However, conventional ADP algorithms struggle to simultaneously manage state and control input constraints during the search for the optimal solution. In real-world applications, another critical and common issue is the presence of external disturbances, where disturbances that cause the control object to deviate from the safe region must be constrained while seeking an optimal control policy. Bearing these factors in mind, this study presents a novel ADP scheme for solving optimal control problems of discrete-time systems, taking into account state and control constraints as well as the impact of disturbances. Moreover, the convergence analysis of the proposed SADP scheme is provided, offering a powerful theoretical foundation for guaranteeing the safety and feasibility of the controlled system during operation.

A New Approach for a Class of Continuous-Time Chaotic Systems Optimal Control by Online ADP Algorithm

Adaptive Dynamic Programming for a Class of Complex-Valued Nonlinear Systems

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

A New Approach to Finite-Horizon Optimal Control for Discrete-Time Affine Nonlinear Systems via a Pseudolinear Method

Robust Optimal Control For Time-Delay Systems With Dynamic Uncertainties Via Adp

Online Adaptive Optimal Control for Continuous-Time Nonlinear Systems with Completely Unknown Dynamics.

Constrained Online Optimal Control for Continuous-Time Nonlinear Systems Using Neuro-Dynamic Programming

Robust ADP Design for Continuous-Time Nonlinear Systems with Output Constraints

Near Optimal Neural Network-based Output Feedback Control of Affine Nonlinear Discrete-Time Systems

Self-Triggered Approximate Optimal Neuro-Control for Nonlinear Systems Through Adaptive Dynamic Programming

Event-triggered optimal decentralized control for stochastic interconnected nonlinear systems via adaptive dynamic programming

Control Uncertain Continuous-Time Chaotic Dynamical System

A neural-network-based online optimal control approach for nonlinear robust decentralized stabilization

ADP-Based Optimal Control for Discrete-Time Systems With Safe Constraints and Disturbances

Optimal Tracking Control for a Class of Unknown Discrete-time Systems with Actuator Saturation Via Data-based ADP Algorithm

Neural-network-based Approach to Finite-Time Optimal Control for a Class of Unknown Nonlinear Systems

Output Constrained Adaptive Dynamic Programming For Continuous-Time Nonlinear Systems

Data-based Self-learning Optimal Control: Research Progress and Prospects

Value Iteration Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems

Discrete-Time Self-Learning Parallel Control

Optimal targeting of nonlinear chaotic systems using a novel evolutionary computing strategy