Abstract:In this paper, a novel adaptive dynamic programming (ADP)-based optimal control method is developed for discrete-time systems subject to constraints and disturbances. Particularly, a safe policy iteration scheme is designed to handle state and input constraints, including both hard and soft constraints, by converting the original policy improvement strategy into a constrained optimization problem with a prescribed state cost function. After that, an actor-critic-disturbance framework is introduced to address the constrained optimal control problem. The robust safety against disturbances is treated as a two-player zero-sum game, where the actor and disturbance neural networks are used to approximate the optimal control input and the disturbance policy, respectively. The convergence property of the proposed algorithm is analyzed, and the multi-step version of the proposed ADP scheme is derived based on this property. Simulation results are demonstrated and discussed to validate the effectiveness and performance of the proposed method. Note to Practitioners—Addressing constraints in optimal control problems is essential for guaranteeing the safe operation of controlled systems. However, conventional ADP algorithms struggle to simultaneously manage state and control input constraints during the search for the optimal solution. In real-world applications, another critical and common issue is the presence of external disturbances, where disturbances that cause the control object to deviate from the safe region must be constrained while seeking an optimal control policy. Bearing these factors in mind, this study presents a novel ADP scheme for solving optimal control problems of discrete-time systems, taking into account state and control constraints as well as the impact of disturbances. Moreover, the convergence analysis of the proposed SADP scheme is provided, offering a powerful theoretical foundation for guaranteeing the safety and feasibility of the controlled system during operation.

Robust ADP-based solution of a class of nonlinear multi-agent systems with input saturation and collision avoidance constraints

Distributed Robust Event-Triggered Adaptive Control for a Class of Uncertain Nonlinear Multi-Agent Systems with Actuator Saturation

Robust Adaptive Control of Uncertain Nonlinear Systems in the Presence of Input Saturation and External Disturbance.

Data-Driven Event-Triggered Adaptive Dynamic Programming Control for Nonlinear Systems with Input Saturation.

Robust ADP Design for Continuous-Time Nonlinear Systems with Output Constraints

Optimal tracking control for unknown nonlinear systems with uncertain input saturation: A dynamic event-triggered ADP algorithm

State consensus cooperative control for a class of nonlinear multi-agent systems with output constraints via ADP approach

Near optimal tracking control of a class of nonlinear systems and an experimental comparison

ADP-Based Optimal Control for Discrete-Time Systems With Safe Constraints and Disturbances

Robust adaptive dynamic programming for optimal nonlinear control design

Robust Sliding Mode Adaptive Dynamic Programming Control for Partially Unknown Second-order Nonlinear Systems

Robust Optimal Control For Time-Delay Systems With Dynamic Uncertainties Via Adp

ADP-Based Robust Tracking Control for a Class of Nonlinear Systems With Unmatched Uncertainties

Online optimal consensus control of unknown linear multi-agent systems via time-based adaptive dynamic programming

Distributed optimal consensus control for nonlinear multi-agent systems with input saturation based on event-triggered adaptive dynamic programming method

Robust adaptive finite-time containment control for nonlinear multi-agent systems with unknown input saturations

Adaptive optimized consensus control for a class of nonlinear multi-agent systems with asymmetric input saturation constraints and hybrid faults

Command Filtered Robust Adaptive NN Control for a Class of Uncertain Strict-Feedback Nonlinear Systems under Input Saturation

Robust adaptive formation control for nonlinear multi-agent systems with range constraints

Guaranteed cost tracking control of constrained input nonlinear uncertain systems using event-based ADP