Abstract:We study online control of time-varying linear systems with unknown dynamics in the nonstochastic control model. At a high level, we demonstrate that this setting is \emph{qualitatively harder} than that of either unknown time-invariant or known time-varying dynamics, and complement our negative results with algorithmic upper bounds in regimes where sublinear regret is possible. More specifically, we study regret bounds with respect to common classes of policies: Disturbance Action (SLS), Disturbance Response (Youla), and linear feedback policies. While these three classes are essentially equivalent for LTI systems, we demonstrate that these equivalences break down for time-varying systems. We prove a lower bound that no algorithm can obtain sublinear regret with respect to the first two classes unless a certain measure of system variability also scales sublinearly in the horizon. Furthermore, we show that offline planning over the state linear feedback policies is NP-hard, suggesting hardness of the online learning problem. On the positive side, we give an efficient algorithm that attains a sublinear regret bound against the class of Disturbance Response policies up to the aforementioned system variability term. In fact, our algorithm enjoys sublinear \emph{adaptive} regret bounds, which is a strictly stronger metric than standard regret and is more appropriate for time-varying systems. We sketch extensions to Disturbance Action policies and partial observation, and propose an inefficient algorithm for regret against linear state feedback policies.

Closed-Loop Finite-Time Analysis of Suboptimal Online Control

On the Finite-Time Behavior of Suboptimal Linear Model Predictive Control

Closed-loop Analysis of ADMM-based Suboptimal Linear Model Predictive Control

An Efficient Optimal Control Method for Open-Loop Transient Stability Emergency Control

Online convex optimization for robust control of constrained dynamical systems

Adaptive Optimal Control for a Class of Continuous-Time Affine Nonlinear Systems with Unknown Internal Dynamics

Online Adaptive Optimal Control for Continuous-Time Nonlinear Systems with Completely Unknown Dynamics.

Suboptimal Control of Linear Systems with State Time-Delay

Nearly optimal stabilization of unknown continuous-time nonlinear systems: A new parallel control approach

Suboptimality analysis of receding horizon quadratic control with unknown linear systems and its applications in learning-based control

Adaptive Optimal Control with Guaranteed Convergence Rate for Continuous-Time Linear Systems with Completely Unknown Dynamics.

Time-Series-Informed Closed-loop Learning for Sequential Decision Making and Control

An Online Learning Analysis of Minimax Adaptive Control

Online Control of Unknown Time-Varying Dynamical Systems

Online Optimal Regulation and Tracking Control of Nonlinear Discrete-Time System with Control Constraints

Adaptive Fuzzy Inverse Optimal Fixed-Time Control of Uncertain Nonlinear Systems

Finite-time Stabilization for a Class of Nonlinear Systems Via Optimal Control

An Online Algorithm for Solving Feedback Optimal Control Problems with Partial Observations

Finite-Time Optimal Control for a Class of Lower-Triangular Nonlinear Systems

Online Approximate Optimal Control for Affine Non-Linear Systems with Unknown Internal Dynamics Using Adaptive Dynamic Programming

Introduction to Online Nonstochastic Control