Abstract:The policy gradient adaptive dynamic programming (PGADP) technique has gained recognition as an effective approach for optimizing the performance of nonlinear systems. Nonetheless, existing PGADP algorithms often demand a substantial volume of expensive or potentially risky interaction data with the system. Moreover, the utilization of neural networks in these algorithms can result in suboptimal learning efficiency and unstable training procedures. To address these challenges, a novel algorithm, referred to as OptNet-PGADP, has been introduced. This algorithm integrates an initially tailored control policy based on OptNet to tackle the optimization of control problems in discrete-time nonlinear systems. The OptNet-PGADP algorithm operates through a two-step process. Initially, the input–output trajectory of the system is computed using the nonlinear model predictive control (NMPC) method. Subsequently, an initial admissible control policy is acquired through OptNet. This policy is iteratively enhanced using the PGADP algorithm to attain the optimal controller. The resulting closed-loop control policy can be readily deployed in real-time applications. The implementation of the algorithm employs OptNet for the actor network and integrates an experience replay mechanism to bolster the controller’s learning efficiency. Furthermore, a convergence and optimality analysis of the algorithm is included. Simulation and experimental results conducted on two nonlinear systems conclusively demonstrate that the approach outperforms traditional PGADP and NMPC algorithms. These findings underscore the efficacy of OptNet-PGADP in mitigating the constraints of current methods and achieving superior control performance for nonlinear systems.

A brief survey on nonlinear control using adaptive dynamic programming under engineering-oriented complexities

Adaptive Dynamic Programming for Control: A Survey and Recent Advances

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Data-Driven Event-Triggered Adaptive Dynamic Programming Control for Nonlinear Systems with Input Saturation.

Adaptive Dynamic Programming for a Class of Complex-Valued Nonlinear Systems

Robust adaptive dynamic programming for optimal nonlinear control design

Adaptive Dynamic Programming for Nonaffine Nonlinear Optimal Control Problem with State Constraints

Optimization-based Adaptive Neural Sliding Mode Control for Nonlinear Systems with Fast and Accurate Response under State and Input Constraints

Robust ADP Design for Continuous-Time Nonlinear Systems with Output Constraints

Optimal control of nonlinear system based on deterministic policy gradient with eligibility traces

ADP-Based Model Reference Adaptive Control Design for Unknown Discrete-Time Nonlinear Systems

Event-triggered optimal decentralized control for stochastic interconnected nonlinear systems via adaptive dynamic programming

Output Constrained Adaptive Dynamic Programming For Continuous-Time Nonlinear Systems

Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems with Trajectory-Based Initial Control Policy

Adaptive dynamic programming for optimal control of discrete‐time nonlinear system with state constraints based on control barrier function

A Parallel Framework of Adaptive Dynamic Programming Algorithm with Off-Policy Learning.

Optimal prescribed performance tracking control of nonlinear motor driven systems via adaptive dynamic programming

Observed-based Event-Triggered Control for Nonlinear Systems with Disturbances Using Adaptive Dynamic Programming

Reinforcement learning and adaptive/approximate dynamic programming: A survey from theory to applications in multi-agent systems

Adaptive Dynamic Programming Based Event-Triggered Control for Unknown Continuous-Time Nonlinear Systems with Input Constraints

Online Adaptive Optimal Control for Continuous-Time Nonlinear Systems with Completely Unknown Dynamics.