Abstract:This paper presents a one-shot learning approach with performance and robustness guarantees for the linear quadratic regulator (LQR) control of stochastic linear systems. Even though data-based LQR control has been widely considered, existing results suffer either from data hungriness due to the inherently iterative nature of the optimization formulation (e.g., value learning or policy gradient reinforcement learning algorithms) or from a lack of robustness guarantees in one-shot non-iterative algorithms. To avoid data hungriness while ensuing robustness guarantees, an adaptive dynamic programming formalization of the LQR is presented that relies on solving a Bellman inequality. The control gain and the value function are directly learned by using a control-oriented approach that characterizes the closed-loop system using data and a decision variable from which the control is obtained. This closed-loop characterization is noise-dependent. The effect of the closed-loop system noise on the Bellman inequality is considered to ensure both robust stability and suboptimal performance despite ignoring the measurement noise. To ensure robust stability, it is shown that this system characterization leads to a closed-loop system with multiplicative and additive noise, enabling the application of distributional robust control techniques. The analysis of the suboptimality gap reveals that robustness can be achieved without the need for regularization or parameter tuning. The simulation results on the active car suspension problem demonstrate the superiority of the proposed method in terms of robustness and performance gap compared to existing methods.

Minimax Q-learning Control for Linear Systems Using the Wasserstein Metric

Data-drivenDistributionallyRobustOptimal Stochastic ControlUsing theWassersteinMetric

Minimax control of ambiguous linear stochastic systems using the Wasserstein metric

Minimax Optimal Control of Uncertain Quasi-Integrable Hamiltonian Systems with Time-Delayed Bounded Feedback

Distributional robustness in minimax linear quadratic control with Wasserstein distance

A Minimax Optimal Control Strategy for Uncertain Quasi-Hamiltonian Systems

Stochastic Minimax Vibration Control for Uncertain Nonlinear Quasi-Hamiltonian Systems with Noisy Observations

A Minimax Stochastic Optimal Control for Bounded-Uncertain Systems

A Minimax Optimal Control Strategy for Partially Observable Uncertain Quasi-Hamiltonian Systems

STOCHASTIC OPTIMAL CONTROL OF UNCERTAIN QUASI-HAMILTONIAN SYSTEMS

A Q-Learning Algorithm for Discrete-Time Linear-Quadratic Control with Random Parameters of Unknown Distribution: Convergence and Stabilization

On Stochastic Optimal Control of Partially Observable Nonlinear Quasi Hamiltonian Systems

Stochastic Minimax Optimal Time-Delay State Feedback Control of Uncertain Quasi-Integrable Hamiltonian Systems

Risk-Constrained Control of Mean-Field Linear Quadratic Systems

A nonlinear stochastic optimal bounded control using stochastic maximum principle

Adaptive dynamic programming and distributionally robust optimal control of linear stochastic system using the Wasserstein metric

Brief paper: Stochastic minimax control for stabilizing uncertain quasi- integrable Hamiltonian systems

Average Cost Optimal Control of Stochastic Systems Using Reinforcement Learning

Stochastic Optimal Linear Quadratic Regulation Control of Discrete-time Systems with Delay and Quadratic Constraints

Value iteration for LQR control of unknown stochastic-parameter linear systems

Direct Data-Driven Discounted Infinite Horizon Linear Quadratic Regulator with Robustness Guarantees