An Online Learning Analysis of Minimax Adaptive Control

Venkatraman Renganathan,Andrea Iannelli,Anders Rantzer

2023-09-11

Abstract:We present an online learning analysis of minimax adaptive control for the case where the uncertainty includes a finite set of linear dynamical systems. Precisely, for each system inside the uncertainty set, we define the model-based regret by comparing the state and input trajectories from the minimax adaptive controller against that of an optimal controller in hindsight that knows the true dynamics. We then define the total regret as the worst case model-based regret with respect to all models in the considered uncertainty set. We study how the total regret accumulates over time and its effect on the adaptation mechanism employed by the controller. Moreover, we investigate the effect of the disturbance on the growth of the regret over time and draw connections between robustness of the controller and the associated regret rate.

Systems and Control

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to analyze minimax adaptive control through online learning in the presence of uncertainties in linear dynamical systems with a finite set. Specifically, for each system in the uncertainty set, the author defines model - based regret, that is, comparing the state and input trajectories of the minimax adaptive controller with those of the optimal controller that knows the true dynamics a posteriori. Then, the total regret is defined as the worst - case model - based regret with respect to all models in the considered uncertainty set. The paper studies the accumulation of total regret over time and its impact on the adaptive mechanisms adopted by the controller. In addition, it also explores the influence of disturbances on the growth of regret over time and establishes the connection between the robustness of the controller and the associated regret rate.

The main contributions of the paper are as follows:
1. Define model - based regret corresponding to a specific model in the uncertainty set, and total regret as the worst - case model - based regret for any model in the uncertainty set.
2. Construct an adversarial disturbance strategy that can provably prevent the minimax adaptive controller from learning the true dynamics (Theorem 1).
3. Although it may be difficult to identify the true dynamics, show that the minimax adaptive controller has a sub - linear regret rate with respect to the best a posteriori H∞ controller (Theorem 2).

Through these analyses, the author aims to improve the understanding of the role of adaptive mechanisms and the impact of adversarial disturbances on regret.