Abstract:Dynamic programming (DP) is an approach to computing the optimal control policy over time under nonlinearity and uncertainty by employing the principle of optimality introduced by Richard Bellman. Instead of enumerating all possible control sequences, dynamic programming only searches admissible state and/or action values that satisfy the principle of optimality. Therefore, the computation complexity can be much improved over the direct enumeration method. However, the computational efforts and the data storage requirement increase exponentially with the dimensionality of the system, which are reflected in the three curses: the state space, the observation space, and the action space. Thus, the traditional DP approach was limited to solving small size problems. This paper aims at providing an overview of latest development of a class of approximate/adaptive dynamic programming algorithms including those applicable to continuous state and continuous control problems. The paper will especially review direct heuristic dynamic programming (direct (HDP), its design and applications, which include large and complex continuous state and control problems. In addition to the basic principle of direct HDP, the paper includes two application studies of the direct HDP - one is when it is used in a nonlinear tracking problem, and the other is on a power grid coordination control problem based on China southern network.

Near-optimal control laws based on Heuristic Dynamic Programming iteration algorithm

Optimal Control for Hybrid Systems Based on Mixed Dynamic Programming

Intelligent Optimal Control of Constrained Nonlinear Systems Via Receding-Horizon Heuristic Dynamic Programming

Approximately Optimal Control of Discrete-Time Nonlinear Switched Systems Using Globalized Dual Heuristic Programming

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Multi-step Heuristic Dynamic Programming for Optimal Control of Nonlinear Discrete-Time Systems.

Near Optimal Control for a Class of Stochastic Hybrid Systems.

H<inf>&#x221E;</inf> Control of Unknown Discrete-Time Nonlinear Systems with Control Constraints Using Adaptive Dynamic Programming

Adaptive Dynamic Programming for a Class of Complex-Valued Nonlinear Systems

Heuristic dynamic programming-based H∞ control of discrete-time affine nonlinear systems

Optimal Control of Unknown Discrete-Time Nonlinear Systems with Constrained Inputs Using GDHP Technique

H ∞ optimal control of unknown linear systems by adaptive dynamic programming with applications to time‐delay systems

Optimal Control of Hybrid Systems Based on Dynamic Programming

Iterative GDHP-based Approximate Optimal Tracking Control for a Class of Discrete-Time Nonlinear Systems

Adaptive neural event‐triggered near‐optimal control for affined uncertain nonlinear discrete‐time system

Neural-network-based Optimal Control for Discrete-Time Nonlinear Systems Using General Value Iteration

Learning-Based N-Step Heuristic Dynamic Programming for Affine Nonlinear Optimal Regulation

Approximate dynamic programming for continuous state and control problems

Discrete‐Time Optimal Control of State‐Constrained Nonlinear Systems Using Approximate Dynamic Programming

Robust Optimal Control For Time-Delay Systems With Dynamic Uncertainties Via Adp

Adaptive Multi-Step Evaluation Design With Stability Guarantee for Discrete-Time Optimal Learning Control