Abstract:In this paper, to tackle the optimal control problem, we propose a $\mathcal{Q}$ -Learning approach for continuous-time nonlinear systems without any dynamic information. Primarily, the Hamiltonian and optimum cost functions are utilized to articulate the $\mathcal{Q}$ -function of continuous-time affine systems. To reduce the dependence of algorithms on system information, a novel $\mathcal{Q}$ -Learning approach is derived to obtain optimal solutions of nonlinear continuous-time systems without requiring knowledge of either the drift information $p(x)$ or input gain $q(x)$ . To implement this approach, critic and actor neural networks can be iterated alternately using an integral reinforcement learning method to estimate the $\mathcal{Q}$ -function. Furthermore, all signals in closed-loop system are demonstrated to be ultimate uniform bounded (UUB). It is worth noting that there exist rare literatures focused on the optimal control problem of continuous-time nonlinear uncertain systems via the $\mathcal{Q}$ -Learning for actor/critic networks iteration. Finally, two simulations are used to confirm the effectiveness of the proposed algorithm. Note to Practitioners —Nonlinear continuous-time systems, being ubiquitous in engineering practice, are widely employed due to their versatility and effectiveness. Aiming at such systems, a $\mathcal{Q}$ -learning approach with optimal feature is proposed to strengthen control efficiency while reduce costs. However, it is well known that accurately capturing all the dynamic information of the system is a formidable task in practical operation. This defect inevitably weakens the feasibility of model-based control algorithms. Since the $\mathcal{Q}$ -learning algorithm presented in this paper does not require any dynamic knowledge of systems, it is promising enabler in enhancing the effectiveness and flexibility of engineering activities.

Reinforcement learning-based optimal control of uncertain nonlinear systems

Reinforcement Learning-Based Control for a Class of Nonlinear Systems with Unknown Control Directions

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Reinforcement Learning for Adaptive Optimal Control of Unknown Continuous-Time Nonlinear Systems with Input Constraints.

Reinforcement Learning for Robust Adaptive Control of Partially Unknown Nonlinear Systems Subject to Unmatched Uncertainties

Robust control scheme for a class of uncertain nonlinear systems with completely unknown dynamics using data-driven reinforcement learning method.

Off‐policy reinforcement learning algorithm for robust optimal control of uncertain nonlinear systems

NN Adaptive Optimal Tracking Control for a Class of Uncertain Nonstrict Feedback Nonlinear Systems

Adaptive Critic Learning-Based Robust Control of Systems with Uncertain Dynamics

Robust adaptive neural network optimal control for a class of uncertain nonlinear systems

Optimal Robust Control of Nonlinear Uncertain System Via Off-Policy Integral Reinforcement Learning

A learning optimal control scheme for robust stabilization of a class of uncertain nonlinear systems

Adaptive Optimal Robust Control for Uncertain Nonlinear Systems Using Neural Network Approximation in Policy Iteration

Reinforcement Learning-Based Optimal Stabilization for Unknown Nonlinear Systems Subject to Inputs With Uncertain Constraints

Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming

Reinforcement Learning for Robust Neuro-Control of Constrained Nonlinear Systems

Dynamic Compensator-Based Near-Optimal Control for Unknown Nonaffine Systems Via Integral Reinforcement Learning

Optimal Robust Control of Nonlinear Systems with Unknown Dynamics via NN Learning with Relaxed Excitation

An Adaptive Dynamic Programming Algorithm to Solve Optimal Control of Uncertain Nonlinear Systems

Adaptive Dynamic Programming Based Robust Control of Nonlinear Systems with Unmatched Uncertainties

Optimal Control for Continuous-Time Unknown Nonlinear Affine Systems: A $\Mathcal{q}$-Learning Approach