Abstract:The policy gradient adaptive dynamic programming (PGADP) technique has gained recognition as an effective approach for optimizing the performance of nonlinear systems. Nonetheless, existing PGADP algorithms often demand a substantial volume of expensive or potentially risky interaction data with the system. Moreover, the utilization of neural networks in these algorithms can result in suboptimal learning efficiency and unstable training procedures. To address these challenges, a novel algorithm, referred to as OptNet-PGADP, has been introduced. This algorithm integrates an initially tailored control policy based on OptNet to tackle the optimization of control problems in discrete-time nonlinear systems. The OptNet-PGADP algorithm operates through a two-step process. Initially, the input–output trajectory of the system is computed using the nonlinear model predictive control (NMPC) method. Subsequently, an initial admissible control policy is acquired through OptNet. This policy is iteratively enhanced using the PGADP algorithm to attain the optimal controller. The resulting closed-loop control policy can be readily deployed in real-time applications. The implementation of the algorithm employs OptNet for the actor network and integrates an experience replay mechanism to bolster the controller’s learning efficiency. Furthermore, a convergence and optimality analysis of the algorithm is included. Simulation and experimental results conducted on two nonlinear systems conclusively demonstrate that the approach outperforms traditional PGADP and NMPC algorithms. These findings underscore the efficacy of OptNet-PGADP in mitigating the constraints of current methods and achieving superior control performance for nonlinear systems.

Optimal consensus control for multi-agent systems: Multi-step policy gradient adaptive dynamic programming method

Data-Based Optimal Consensus Control for Multiagent Systems With Policy Gradient Reinforcement Learning

Online optimal consensus control of unknown linear multi-agent systems via time-based adaptive dynamic programming

A DDPG-based solution for optimal consensus of continuous-time linear multi-agent systems

Distributed Optimal Consensus Control for Nonlinear Multiagent System with Unknown Dynamic.

Deterministic Policy Gradient Adaptive Dynamic Programming for Model-Free Optimal Control

Finite-horizon Approximate Optimal Consensus Control for Discrete-time Nonlinear Multi-Agent Systems with Unknown Dynamics

Twin Deterministic Policy Gradient Adaptive Dynamic Programming for Optimal Control of Affine Nonlinear Discrete-time Systems

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Policy Gradient Adaptive Dynamic Programming for Model-Free Multi-Objective Optimal Control

Optimal Scaled Average Consensus Control of Discrete-Time Multi-Agent Systems Based on Adaptive Dynamic Programming

Dynamic Event-Triggered Consensus Control for Multi-Agent Systems Using Adaptive Dynamic Programming

Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems with Trajectory-Based Initial Control Policy

Policy Gradient Adaptive Dynamic Programming for Data-Based Optimal Control.

Online Optimal Consensus Control Based on PI Algorithm for Nonlinear Multi-Agent System with Completely Unknown Dynamics

Data-based Optimal Control for Discrete-time Systems Via Deep Deterministic Policy Gradient Adaptive Dynamic Programming

Distributed Optimal Control of Nonlinear System Based on Policy Gradient with External Disturbance

Multi-step Heuristic Dynamic Programming for Optimal Control of Nonlinear Discrete-Time Systems.

Optimal Consensus Control Design for Multiagent Systems With Multiple Time Delay Using Adaptive Dynamic Programming

Optimal consensus control for unknown second-order multi-agent systems: Using model-free reinforcement learning method

Optimal consensus model-free control for multi-agent systems subject to input delays and switching topologies