Abstract:Reinforcement learning has proved to be a powerful tool to solve optimal control problems over the past few years. However, the data-based constrained optimal control problem of nonaffine nonlinear discrete-time systems has rarely been studied yet. To solve this problem, an adaptive optimal control approach is developed by using the value iteration-based Q-learning (VIQL) with the critic-only structure. Most of the existing constrained control methods require the use of a certain performance index and only suit for linear or affine nonlinear systems, which is unreasonable in practice. To overcome this problem, the system transformation is first introduced with the general performance index. Then, the constrained optimal control problem is converted to an unconstrained optimal control problem. By introducing the action-state value function, i.e., Q-function, the VIQL algorithm is proposed to learn the optimal Q-function of the data-based unconstrained optimal control problem. The convergence results of the VIQL algorithm are established with an easy-to-realize initial condition . To implement the VIQL algorithm, the critic-only structure is developed, where only one neural network is required to approximate the Q-function. The converged Q-function obtained from the critic-only VIQL method is employed to design the adaptive constrained optimal controller based on the gradient descent scheme. Finally, the effectiveness of the developed adaptive control method is tested on three examples with computer simulation.

Hybrid Q-learning for Data-Based Optimal Control of Non-Linear Switching System

Hybrid Reinforcement Learning for Optimal Control of Non-Linear Switching System

Optimal Control for Hybrid Systems Based on Mixed Dynamic Programming

A nonlinear predictive control algorithm based on fuzzy online modeling and discrete optimization

On Modeling and Optimal Control of a Class of Hybrid Systems

A hybrid model-based optimal control method for nonlinear systems using simultaneous dynamic optimization strategies

Fuzzy Optimal Control for a Class of Discrete-Time Switched Nonlinear Systems

An Optimal Tracking Control Method with Q-learning for Discrete-time Linear Switched System

Adaptive Constrained Optimal Control Design for Data-Based Nonlinear Discrete-Time Systems With Critic-Only Structure

Near Optimal Control for a Class of Stochastic Hybrid Systems.

Data-based Optimal Tracking of Autonomous Nonlinear Switching Systems.

Approximately Optimal Control of Discrete-Time Nonlinear Switched Systems Using Globalized Dual Heuristic Programming

A Combined Policy Gradient and Q-learning Method for Data-driven Optimal Control Problems

Optimal Control Of A Class Of Non-Linear Discrete-Continuous Hybrid Systems

Data-Driven Near-Optimal Control of Nonlinear Systems Over Finite Horizon

H∞$$ {h}_{\infty } $$ Optimal Output Tracking Control for Markov Jump Systems: A Reinforcement Learning‐based Approach

Fuzzy $H_{\infty }$ Control of Discrete-Time Nonlinear Markov Jump Systems via a Novel Hybrid Reinforcement $Q$-Learning Method

An ADDHP-based Q-learning Algorithm for Optimal Tracking Control of Linear Discrete-Time Systems with Unknown Dynamics

A hybrid learning method for system identification and optimal control

Data-driven Adaptive Optimal Control for Discrete-Time Linear Time-Invariant Systems

Optimal Control of Switched Hybrid Systems