Abstract:We propose a new risk-constrained formulation of the classical Linear Quadratic (LQ) stochastic control problem for general partially-observed systems. Our framework is motivated by the fact that the risk-neutral LQ controllers, although optimal in expectation, might be ineffective under relatively infrequent, yet statistically significant extreme events. To effectively trade between average and extreme event performance, we introduce a new risk constraint, which explicitly restricts the total expected predictive variance of the state penalty by a user-prescribed level. We show that, under certain conditions on the process noise, the optimal risk-aware controller can be evaluated explicitly and in closed form. In fact, it is affine relative to the minimum mean square error (mmse) state estimate. The affine term pushes the state away from directions where the noise exhibits heavy tails, by exploiting the third-order moment~(skewness) of the noise. The linear term regulates the state more strictly in riskier directions, where both the prediction error (conditional) covariance and the state penalty are simultaneously large; this is achieved by inflating the state penalty within a new filtered Riccati difference equation. We also prove that the new risk-aware controller is internally stable, regardless of parameter tuning, in the special cases of i) fully-observed systems, and ii) partially-observed systems with Gaussian noise. The properties of the proposed risk-aware LQ framework are lastly illustrated via indicative numerical examples.

Regret Bounds for Episodic Risk-Sensitive Linear Quadratic Regulator

Learning Decentralized Linear Quadratic Regulators with [math] Regret

Rate-matching the regret lower-bound in the linear quadratic regulator with unknown dynamics

Almost Surely $\sqrt{T}$ Regret Bound for Adaptive LQR

Learning Decentralized Linear Quadratic Regulators with $\sqrt{T}$ Regret

Stronger Regret Bounds for Safe Online Reinforcement Learning in the Linear Quadratic Regulator

Online Actuator Selection and Controller Design for Linear Quadratic Regulation with Unknown System Model

Sublinear Regret for a Class of Continuous-Time Linear--Quadratic Reinforcement Learning Problems

Regret Analysis for Risk-aware Linear Quadratic Control

Fully Adaptive Regret-Guaranteed Algorithm for Control of Linear Quadratic Systems

Risk-Constrained Linear-Quadratic Regulators

On Adaptive Linear-Quadratic Regulators

Regret Bounds for Risk-sensitive Reinforcement Learning with Lipschitz Dynamic Risk Measures

Linear Quadratic Control with Risk Constraints

Controlling Unknown Linear Dynamics with Almost Optimal Regret

Regret Analysis of Online LQR Control via Trajectory Prediction and Tracking: Extended Version

Online Linear Quadratic Tracking with Regret Guarantees

An Iterative Riccati Algorithm for Online Linear Quadratic Control

Settling Constant Regrets in Linear Markov Decision Processes

Regret Lower Bounds for Learning Linear Quadratic Gaussian Systems

Episodic Linear Quadratic Regulators with Low-rank Transitions