Abstract:In the past decade, adaptive critic designs (ACDs), including heuristic dynamic programming (HDP), dual heuristic programming (DHP), and their action-dependent ones, have been widely studied to realize online learning control of dynamical systems. However, because neural networks with manually designed features are commonly used to deal with continuous state and action spaces, the generalization capability and learning efficiency of previous ACDs still need to be improved. In this paper, a novel framework of ACDs with sparse kernel machines is presented by integrating kernel methods into the critic of ACDs. To improve the generalization capability as well as the computational efficiency of kernel machines, a sparsification method based on the approximately linear dependence analysis is used. Using the sparse kernel machines, two kernel-based ACD algorithms, that is, kernel HDP (KHDP) and kernel DHP (KDHP), are proposed and their performance is analyzed both theoretically and empirically. Because of the representation learning and generalization capability of sparse kernel machines, KHDP and KDHP can obtain much better performance than previous HDP and DHP with manually designed neural networks. Simulation and experimental results of two nonlinear control problems, that is, a continuous-action inverted pendulum problem and a ball and plate control problem, demonstrate the effectiveness of the proposed kernel ACD methods.

Kernel-based Adaptive Critic Designs for Optimal Control of Nonlinear Discrete-time System

Online Learning Control Using Adaptive Critic Designs with Sparse Kernel Machines

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

Tracking Control of Affine Nonlinear Discrete-Time Systems Based on Gaussian-kernel-based ADP

Online Sparse Kernel Learning-Based Adaptive Dynamic Programming

Adaptive critic design-based robust neural network control for nonlinear distributed parameter systems with unknown dynamics.

Near Optimal Neural Network-based Output Feedback Control of Affine Nonlinear Discrete-Time Systems

Online Approximate Optimal Control for Affine Non-Linear Systems with Unknown Internal Dynamics Using Adaptive Dynamic Programming

Tracking Control of Discrete-Time Affine Nonlinear Systems Based on Kernel-Hdp Algorithm

On-Line Learning Control for Discrete Nonlinear Systems Via an Improved ADDHP Method

Stable Adaptive Control Using New Critic Designs

Online Adaptation of Kernel Learning Adaptive Predictive Controller

Adaptive Critic Design Based Robust Neural Network Control for a Class of Continuous-Time Nonaffine Nonlinear System

Learning Control of a Bioreactor System Using Kernel-Based Heuristic Dynamic Programming

Adaptive Critic Designs for Optimal Control of Uncertain Nonlinear Systems with Unmatched Interconnections.

Adaptive optimal control for nonlinear discrete-time systems

A Control Algorithm Base on Adaptive Critic Designs

Data-Based On-Line Optimal Control for Unknown Nonlinear Systems Via Adaptive Dynamic Programming Approach

Optimal H∞ Tracking Control of Nonlinear Systems with Zero-Equilibrium-free Via Novel Adaptive Critic Designs.

A Novel Neural Network Discrete‐time Optimal Control Design for Nonlinear Time‐delay Systems Using Adaptive Critic Designs