Abstract:This paper presents a one-shot learning approach with performance and robustness guarantees for the linear quadratic regulator (LQR) control of stochastic linear systems. Even though data-based LQR control has been widely considered, existing results suffer either from data hungriness due to the inherently iterative nature of the optimization formulation (e.g., value learning or policy gradient reinforcement learning algorithms) or from a lack of robustness guarantees in one-shot non-iterative algorithms. To avoid data hungriness while ensuing robustness guarantees, an adaptive dynamic programming formalization of the LQR is presented that relies on solving a Bellman inequality. The control gain and the value function are directly learned by using a control-oriented approach that characterizes the closed-loop system using data and a decision variable from which the control is obtained. This closed-loop characterization is noise-dependent. The effect of the closed-loop system noise on the Bellman inequality is considered to ensure both robust stability and suboptimal performance despite ignoring the measurement noise. To ensure robust stability, it is shown that this system characterization leads to a closed-loop system with multiplicative and additive noise, enabling the application of distributional robust control techniques. The analysis of the suboptimality gap reveals that robustness can be achieved without the need for regularization or parameter tuning. The simulation results on the active car suspension problem demonstrate the superiority of the proposed method in terms of robustness and performance gap compared to existing methods.

Robust guarantees for learning an autoregressive filter

Data-driven Optimal Filtering for Linear Systems with Unknown Noise Covariances

Control of Nonaffine Nonlinear Discrete-Time Systems Using Reinforcement-Learning-Based Linearly Parameterized Neural Networks

Robust Adaptive Repetitive Learning Control for a Class of Time-Varying Nonlinear Systems with Unknown Control Direction

Robust Adaptive Iterative Learning Control for Discrete-Time Nonlinear Systems With Time-Iteration-Varying Parameters.

Mean-square error constrained approach to robust stochastic iterative learning control

Online Learning of the Kalman Filter with Logarithmic Regret

Direct Data-Driven Discounted Infinite Horizon Linear Quadratic Regulator with Robustness Guarantees

Robust H∞ Filtering for Networked Stochastic Systems with Randomly Occurring Sensor Nonlinearities and Packet Dropouts

On-Line Learning of Linear Dynamical Systems: Exponential Forgetting in Kalman Filters

SLIP: Learning to Predict in Unknown Dynamical Systems with Long-Term Memory

Reinforcement Learning for Linear Exponential Quadratic Gaussian Problem

H_∞ Model-free Reinforcement Learning with Robust Stability Guarantee

Predictive stability filters for nonlinear dynamical systems affected by disturbances

Discrete-Time Robust Iterative Learning Kalman Filtering for Repetitive Processes

Robust Reinforcement Learning for Risk-Sensitive Linear Quadratic Gaussian Control

Dynamic Learning from Neural Network‐based Control for Sampled‐data Strict‐feedback Nonlinear Systems

Robust Kalman filters with unknown covariance of multiplicative noise

Robust Nonlinear Filter for Nonlinear Systems with Multiplicative Noise Uncertainties, Unknown External Disturbances, and Packet Dropouts

Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach