Abstract:In this paper we propose a framework towards achieving two intertwined objectives: (i) equipping reinforcement learning with active exploration and deliberate information gathering, such that it regulates state and parameter uncertainties resulting from modeling mismatches and noisy sensory; and (ii) overcoming the huge computational cost of stochastic optimal control. We approach both objectives by using reinforcement learning to attain the stochastic optimal control law. On one hand, we avoid the curse of dimensionality prohibiting the direct solution of the stochastic dynamic programming equation. On the other hand, the resulting stochastic control inspired reinforcement learning agent admits the behavior of a dual control, namely, caution and probing, that is, regulating the state estimate together with its estimation quality. Unlike exploration and exploitation, caution and probing are employed automatically by the controller in real-time, even after the learning process is concluded. We use the proposed approach on a numerical example of a model that belongs to an emerging class in system identification. We show how, for the dimensionality of the stochastic version of this model, Dynamic Programming is prohibitive, Model Predictive Control requires an expensive nonlinear optimization, and a Linear Quadratic Regulator with the certainty equivalence assumption leads to poor performance and filter divergence, all contrasting our approach which is shown to be both: computationally convenient, stabilizing and of an acceptable performance.

Deep Reinforcement Learning for Online Control of Stochastic Partial Differential Equations

Online identification and control of PDEs via Reinforcement Learning methods

Reinforcement Learning with Function-Valued Action Spaces for Partial Differential Equation Control

Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policies

Interpretable and Efficient Data-driven Discovery and Control of Distributed Systems

Distributed control of partial differential equations using convolutional reinforcement learning

Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach

Online Optimization of Dynamical Systems with Deep Learning Perception

Model-Based Reinforcement Learning Control of Reaction-Diffusion Problems

Finite Horizon Multi-Agent Reinforcement Learning in Solving Optimal Control of State-Dependent Switched Systems

Solving Stochastic Optimal Control Problem Via Stochastic Maximum Principle with Deep Learning Method

Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows

Reinforcement Learning Policies in Continuous-Time Linear Systems

Deep BSDE-ML Learning and Its Application to Model-Free Optimal Control

Data-based Suboptimal Neuro-control Design with Reinforcement Learning for Dissipative Spatially Distributed Processes

Online Policy Iteration Algorithm for Optimal Control of Linear Hyperbolic PDE Systems

A deep learning method for solving stochastic optimal control problems driven by fully-coupled FBSDEs

Deep Reinforcement Learning in Finite-Horizon to Explore the Most Probable Transition Pathway

Deep Graphic FBSDEs for Opinion Dynamics Stochastic Control

A Deep Reinforcement Learning Approach to Improve the Learning Performance in Process Control

Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications