Abstract:In this paper we propose a framework towards achieving two intertwined objectives: (i) equipping reinforcement learning with active exploration and deliberate information gathering, such that it regulates state and parameter uncertainties resulting from modeling mismatches and noisy sensory; and (ii) overcoming the huge computational cost of stochastic optimal control. We approach both objectives by using reinforcement learning to attain the stochastic optimal control law. On one hand, we avoid the curse of dimensionality prohibiting the direct solution of the stochastic dynamic programming equation. On the other hand, the resulting stochastic control inspired reinforcement learning agent admits the behavior of a dual control, namely, caution and probing, that is, regulating the state estimate together with its estimation quality. Unlike exploration and exploitation, caution and probing are employed automatically by the controller in real-time, even after the learning process is concluded. We use the proposed approach on a numerical example of a model that belongs to an emerging class in system identification. We show how, for the dimensionality of the stochastic version of this model, Dynamic Programming is prohibitive, Model Predictive Control requires an expensive nonlinear optimization, and a Linear Quadratic Regulator with the certainty equivalence assumption leads to poor performance and filter divergence, all contrasting our approach which is shown to be both: computationally convenient, stabilizing and of an acceptable performance.

An Observer-Based Reinforcement Learning Solution for Model-Following Problems

Model-Based Robot Learning Control with Uncertainty Directed Exploration

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

A Data-Driven Model-Reference Adaptive Control Approach Based on Reinforcement Learning

Online Multi-Objective Model-Independent Adaptive Tracking Mechanism for Dynamical Systems

Model-based reinforcement learning for infinite-horizon approximate optimal tracking

Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach

Real-Time Measurement-Driven Reinforcement Learning Control Approach for Uncertain Nonlinear Systems

Data-Efficient Off-Policy Learning for Distributed Optimal Tracking Control of HMAS with Unidentified Exosystem Dynamics.

Adaptive Reinforcement Learning Strategy-Based Sliding Mode Control of Uncertain Euler-Lagrange Systems with Prescribed Performance Guarantees: Autonomous Underwater Vehicles-Based Verification

Model-Free Cooperative Optimal Output Regulation for Linear Discrete-Time Multi-Agent Systems Using Reinforcement Learning

Multi-Objective Learning Model Predictive Control

Optimized backstepping consensus control using adaptive observer-critic-actor reinforcement learning for strict-feedback multi-agent systems

Combining hybrid metaheuristic algorithms and reinforcement learning to improve the optimal control of nonlinear continuous-time systems with input constraints

Optimal Tracking Control of Heterogeneous MASs Using Event-Driven Adaptive Observer and Reinforcement Learning

Model-Based Reinforcement Learning Control of Electrohydraulic Position Servo Systems

Model Reduction and Neural Network Adaptive Control of Nonlinear Hydraulic System Based on Extended State Observer

Output Feedback Adaptive Optimal Control of Affine Nonlinear systems with a Linear Measurement Model

Adaptive control of a mechatronic system using constrained residual reinforcement learning

Observer-Based Consensus Control for MASs With Prescribed Constraints via Reinforcement Learning Algorithm

A Hierarchical Distributed Data-Driven Adaptive Learning Control for Nonaffine Nonlinear MASs