Abstract:This paper presents a one-shot learning approach with performance and robustness guarantees for the linear quadratic regulator (LQR) control of stochastic linear systems. Even though data-based LQR control has been widely considered, existing results suffer either from data hungriness due to the inherently iterative nature of the optimization formulation (e.g., value learning or policy gradient reinforcement learning algorithms) or from a lack of robustness guarantees in one-shot non-iterative algorithms. To avoid data hungriness while ensuing robustness guarantees, an adaptive dynamic programming formalization of the LQR is presented that relies on solving a Bellman inequality. The control gain and the value function are directly learned by using a control-oriented approach that characterizes the closed-loop system using data and a decision variable from which the control is obtained. This closed-loop characterization is noise-dependent. The effect of the closed-loop system noise on the Bellman inequality is considered to ensure both robust stability and suboptimal performance despite ignoring the measurement noise. To ensure robust stability, it is shown that this system characterization leads to a closed-loop system with multiplicative and additive noise, enabling the application of distributional robust control techniques. The analysis of the suboptimality gap reveals that robustness can be achieved without the need for regularization or parameter tuning. The simulation results on the active car suspension problem demonstrate the superiority of the proposed method in terms of robustness and performance gap compared to existing methods.

Optimality of Linear Policies for Distributionally Robust Linear Quadratic Gaussian Regulator with Stationary Distributions

Distributionally Robust Linear Quadratic Control

The Power of Linear Controllers in LQR Control

Distributional robustness in minimax linear quadratic control with Wasserstein distance

A Polynomial Chaos Approach to Stochastic LQ Optimal Control: Error Bounds and Infinite-Horizon Results

Risk-Constrained Linear-Quadratic Regulators

Uniform Ergodicity and Ergodic-Risk Constrained Policy Optimization

Policy Gradient Methods for Discrete Time Linear Quadratic Regulator With Random Parameters

On the Volatility of Optimal Control Policies and the Capacity of a Class of Linear Quadratic Regulators

Distributionally Robust LQG control under Distributed Uncertainty

Direct Data-Driven Discounted Infinite Horizon Linear Quadratic Regulator with Robustness Guarantees

Linear Quadratic Regulation and Stabilization of Discrete-Time Systems With Delay and Multiplicative Noise

The Distributionally Robust Infinite-Horizon LQR

Guaranteed Stability Margins for Decentralized Linear Quadratic Regulators

Linear Quadratic Control with Risk Constraints

Wasserstein Distributionally Robust Control of Partially Observable Linear Stochastic Systems

Explicit Solution for Constrained Stochastic Linear-Quadratic Control with Multiplicative Noise

Closing the Gap to Quadratic Invariance: a Regret Minimization Approach to Optimal Distributed Control

Adaptive dynamic programming and distributionally robust optimal control of linear stochastic system using the Wasserstein metric