Abstract:Continuous-time nonlinear optimal control problems hold great promise in real-world applications. After decades of development, reinforcement learning (RL) has achieved some of the greatest successes as a general nonlinear control design method. However, a recent comprehensive analysis of state-of-the-art continuous-time RL (CT-RL) methods, namely, adaptive dynamic programming (ADP)-based CT-RL algorithms, reveals they face significant design challenges due to their complexity, numerical conditioning, and dimensional scaling issues. Despite advanced theoretical results, existing ADP CT-RL synthesis methods are inadequate in solving even small, academic problems. The goal of this work is thus to introduce a suite of new CT-RL algorithms for control of affine nonlinear systems. Our design approach relies on two important factors. First, our methods are applicable to physical systems that can be partitioned into smaller subproblems. This constructive consideration results in reduced dimensionality and greatly improved intuitiveness of design. Second, we introduce a new excitation framework to improve persistence of excitation (PE) and numerical conditioning performance via classical input/output insights. Such a design-centric approach is the first of its kind in the ADP CT-RL community. In this paper, we progressively introduce a suite of (decentralized) excitable integral reinforcement learning (EIRL) algorithms. We provide convergence and closed-loop stability guarantees, and we demonstrate these guarantees on a significant application problem of controlling an unstable, nonminimum phase hypersonic vehicle (HSV).

Reinforcement Learning in Control Theory: A New Approach to Mathematical Problem Solving

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

Some Challenging Problems in Research on Control Theory

Model-Based Reinforcement Learning Control of Reaction-Diffusion Problems

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guarantee

Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach

Steady-State Error Compensation in Reference Tracking and Disturbance Rejection Problems for Reinforcement Learning-Based Control

From self-tuning regulators to reinforcement learning and back again

Residual Reinforcement Learning for Robot Control

Actor-Critic Reinforcement Learning for Control With Stability Guarantee

Reinforcement Learning of Structured Control for Linear Systems with Unknown State Matrix

A General Control-Theoretic Approach for Reinforcement Learning: Theory and Algorithms

Stability-certified reinforcement learning: A control-theoretic perspective

Practical Reinforcement Learning of Stabilizing Economic MPC

Continuous-Time Reinforcement Learning: New Design Algorithms with Theoretical Insights and Performance Guarantees

Reinforcement learning building control approach harnessing imitation learning

Control of nonlinear, complex and black-boxed greenhouse system with reinforcement learning

Pretty darn good control: when are approximate solutions better than approximate models

An Analysis of Reinforcement Learning for Malaria Control

Deep Dive into Model-free Reinforcement Learning for Biological and Robotic Systems: Theory and Practice