Abstract:We consider the infinite-horizon LQR control problem. Motivated by competitive analysis in online learning, as a criterion for controller design we introduce the dynamic regret, defined as the difference between the LQR cost of a causal controller (that has only access to past disturbances) and the LQR cost of the \emph{unique} clairvoyant one (that has also access to future disturbances) that is known to dominate all other controllers. The regret itself is a function of the disturbances, and we propose to find a causal controller that minimizes the worst-case regret over all bounded energy disturbances. The resulting controller has the interpretation of guaranteeing the smallest regret compared to the best non-causal controller that can see the future. We derive explicit formulas for the optimal regret and for the regret-optimal controller for the state-space setting. These explicit solutions are obtained by showing that the regret-optimal control problem can be reduced to a Nehari extension problem that can be solved explicitly. The regret-optimal controller is shown to be linear and can be expressed as the sum of the classical $H_2$ state-feedback law and an $n$-th order controller ($n$ is the state dimension), and its construction simply requires a solution to the standard LQR Riccati equation and two Lyapunov equations. Simulations over a range of plants demonstrate that the regret-optimal controller interpolates nicely between the $H_2$ and the $H_\infty$ optimal controllers, and generally has $H_2$ and $H_\infty$ costs that are simultaneously close to their optimal values. The regret-optimal controller thus presents itself as a viable option for control systems design.

Regret-Guaranteed Safe Switching with Minimum Cost: LQR Setting with Unknown Dynamics

Learn and Control while Switching: with Guaranteed Stability and Sublinear Regret

Joint Stabilization and Regret Minimization through Switching in Over-Actuated Systems (extended version)

Online switching control with stability and regret guarantees

Asynchronous Guaranteed Cost Control of 2-D Markov Jump Roesser Systems

Performance-Guaranteed Fault-Tolerant Control for Uncertain Nonlinear Systems via Learning-Based Switching Scheme

Safe and Efficient Switching Mechanism Design for Uncertified Linear Controller

Robust Control for Switched Systems Subject to Input Saturation and Parametric Uncertainties.

Regret-Optimal LQR Control

Fuzzy Optimal Control for a Class of Discrete-Time Switched Nonlinear Systems

Ensuring the Safety of Uncertified Linear State-Feedback Controllers via Switching

Logic Switching Based Online Periodic Adaptive Learning Control Algorithm Dealing with Unknown Period and Bound of the Uncertain Parameter

Safe and Efficient Switching Controller Design for Partially Observed Linear-Gaussian Systems.

Robust Stability of Switched Nonlinear Systems With Switching Uncertainties.

Guaranteed Cost Control For A Class Of Uncertain Discrete-Time Linear Switched Systems

Almost Surely $\sqrt{T}$ Regret Bound for Adaptive LQR

A System Level Approach to Regret Optimal Control

Adaptive Neural Control for Switched Nonlinear Systems With Unstable Dynamic Uncertainties: A Small Gain-Based Approach

Learning Piecewise Residuals of Control Barrier Functions for Safety of Switching Systems using Multi-Output Gaussian Processes

Robustly exponentially almost sure stabilization of uncertain discrete‐time composite switched systems

Regret Bounds for Episodic Risk-Sensitive Linear Quadratic Regulator