Abstract:In this paper, to tackle the optimal control problem, we propose a $\mathcal{Q}$ -Learning approach for continuous-time nonlinear systems without any dynamic information. Primarily, the Hamiltonian and optimum cost functions are utilized to articulate the $\mathcal{Q}$ -function of continuous-time affine systems. To reduce the dependence of algorithms on system information, a novel $\mathcal{Q}$ -Learning approach is derived to obtain optimal solutions of nonlinear continuous-time systems without requiring knowledge of either the drift information $p(x)$ or input gain $q(x)$ . To implement this approach, critic and actor neural networks can be iterated alternately using an integral reinforcement learning method to estimate the $\mathcal{Q}$ -function. Furthermore, all signals in closed-loop system are demonstrated to be ultimate uniform bounded (UUB). It is worth noting that there exist rare literatures focused on the optimal control problem of continuous-time nonlinear uncertain systems via the $\mathcal{Q}$ -Learning for actor/critic networks iteration. Finally, two simulations are used to confirm the effectiveness of the proposed algorithm. Note to Practitioners —Nonlinear continuous-time systems, being ubiquitous in engineering practice, are widely employed due to their versatility and effectiveness. Aiming at such systems, a $\mathcal{Q}$ -learning approach with optimal feature is proposed to strengthen control efficiency while reduce costs. However, it is well known that accurately capturing all the dynamic information of the system is a formidable task in practical operation. This defect inevitably weakens the feasibility of model-based control algorithms. Since the $\mathcal{Q}$ -learning algorithm presented in this paper does not require any dynamic knowledge of systems, it is promising enabler in enhancing the effectiveness and flexibility of engineering activities.

Learning Instance-Solution Operator For Optimal Control

Optimal Control Operator Perspective and a Neural Adaptive Spectral Method

A Quasi-Sequential Algorithm for PDE-constrained Optimization Based on Space–time Orthogonal Collocation on Finite Elements

An Operator Learning Approach to Nonsmooth Optimal Control of Nonlinear PDEs

Solving PDE-constrained Control Problems Using Operator Learning

Fast algorithm based on reduced precision solution criterion for solving optimal control problem

Optimization methods rooted in optimal control

Optimal Control for Continuous-Time Unknown Nonlinear Affine Systems: A $\Mathcal{q}$-Learning Approach

Successive Chebyshev Pseudospectral Convex Optimization Method for Nonlinear Optimal Control Problems

Discrete‐Time Optimal Control of State‐Constrained Nonlinear Systems Using Approximate Dynamic Programming

Learning-based Optimal Control for Linear Systems with Model Uncertainties

A Convex Optimization Approach to Inverse Optimal Control

A New Smoothing Technique for Bang-Bang Optimal Control Problems

ODE-based Learning to Optimize

Data-driven optimal control via linear transfer operators: A convex approach

A Data-Driven Approach for Inverse Optimal Control

New Methods for Optimal Operational Control of Industrial Processes Using Reinforcement Learning on Two Time Scales

GOPS: A General Optimal Control Problem Solver for Autonomous Driving and Industrial Control Applications

One-shot learning for solution operators of partial differential equations

Bi-Level-Based Inverse Stochastic Optimal Control

Learning-based Optimal Boundary Control for Parabolic Distributed Parameter System with Actuator Dynamics