Abstract:Stable concurrent learning and control of dynamical systems is the subject of adaptive control. Despite being an established field with many practical applications and a rich theory, much of the development in adaptive control for nonlinear systems revolves around a few key algorithms. By exploiting strong connections between classical adaptive nonlinear control techniques and recent progress in optimization and machine learning, we show that there exists considerable untapped potential in algorithm development for both adaptive nonlinear control and adaptive dynamics prediction. We begin by introducing first-order adaptation laws inspired by natural gradient descent and mirror descent. We prove that when there are multiple dynamics consistent with the data, these non-Euclidean adaptation laws implicitly regularize the learned model. Local geometry imposed during learning thus may be used to select parameter vectors -- out of the many that will achieve perfect tracking or prediction -- for desired properties such as sparsity. We apply this result to regularized dynamics predictor and observer design, and as concrete examples, we consider Hamiltonian systems, Lagrangian systems, and recurrent neural networks. We subsequently develop a variational formalism based on the Bregman Lagrangian. We show that its Euler Lagrange equations lead to natural gradient and mirror descent-like adaptation laws with momentum, and we recover their first-order analogues in the infinite friction limit. We illustrate our analyses with simulations demonstrating our theoretical results.

What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper aims to address the issues of stability and convergence in nonlinear parameterized adaptive control and prediction. Specifically, the authors seek to develop new algorithms by leveraging the strong connections between classical adaptive nonlinear control techniques and modern optimization and machine learning, to achieve stable, concurrent learning and control of dynamic systems. ### Main Contributions 1. **Development of Natural Gradient and Mirror Descent Algorithms**: - The authors further develop a class of algorithms inspired by natural gradient and mirror descent, which have applications in learning robotic inertial parameters and geometric convex optimization. - They prove that these algorithms implicitly regularize the system model during the learning process, whether in linear or nonlinear parameterized settings. 2. **Construction of Higher-Order Adaptive Control Algorithms**: - A class of higher-order temporal adaptive control algorithms is constructed, introducing momentum into existing adaptation laws. - It is proven that these new momentum algorithms are stable and globally convergent when dealing with both linear and nonlinear parameterized systems. - Several natural gradient and mirror descent adaptive control algorithms with momentum are designed, which also implicitly regularize the learning model. ### Implicit Regularization - In adaptive control, explicit regularization terms cannot be added arbitrarily as they may affect the system's stability and performance. - The authors' method provides a provably stable and globally convergent way to achieve regularization. - Examples demonstrate the practical utility of these results in dynamic prediction, such as sparse estimation of the Hamiltonian or Lagrangian functions of physical systems, and the estimation of weights in continuous-time recurrent neural network models. ### Related Work - The authors' work continues the tradition of analyzing optimization algorithms from a continuous-time perspective. - A formal framework is introduced for generating higher-order temporal adaptive control algorithms using Bregman Lagrangian and velocity gradient methods. - Recent studies on the implicit bias of optimization algorithms are discussed and connected to implicit regularization in adaptive control. ### Structure Overview - Section 2 introduces the mathematical background of direct adaptive control, including linear and nonlinear parameterized systems, and a review of the velocity gradient method. - Section 3 analyzes the implicit bias of adaptive control algorithms. - Section 4 discusses the design of observers and dynamic predictors, Hamiltonian dynamic prediction, control of Lagrangian systems, and estimation of recurrent neural networks. - Section 5 reviews the Bregman Lagrangian, and Section 6 uses it to design new higher-order adaptation algorithms. - Section 7 discusses the extension of adaptation laws in distributed settings. - Section 8 further extends the non-Euclidean part. - Section 9 demonstrates the research results through simulations. - Section 10 summarizes the entire paper and proposes future research directions.

Implicit Regularization and Momentum Algorithms in Nonlinearly Parameterized Adaptive Control and Prediction

Implicit regularization and momentum algorithms in nonlinear adaptive control and prediction

Immersion and Invariance Adaptive Control of Nonlinearly Parameterized Nonlinear Systems

On adaptive control of nonlinearly parameterized nonlinear systems: Towards a constructive procedure

Modular‐based Adaptive Control of Uncertain Nonlinear Time‐varying Systems with Piecewise Jumping Parameters

Adaptive Controller and Observer Design for A Class of Nonlinear Systems

Momentum-Based Adaptive Laws for Identification and Control

Learning-Based Neural Dynamic Surface Predictive Control for MMC

Accelerated Gradient Approach For Deep Neural Network-Based Adaptive Control of Unknown Nonlinear Systems

Discrete-Time Adaptive Iterative Learning Control for High-Order Nonlinear Systems with Unknown Control Directions

Robust Adaptive Iterative Learning Control for Discrete-Time Nonlinear Systems With Time-Iteration-Varying Parameters.

Adaptive control of A class of nonlinear systems with guaranteed parameter estimation: A concurrent learning based approach

Meta-Learning for Adaptive Control with Automated Mirror Descent

Learning to control non-equilibrium dynamics using local imperfect gradients

Control-Theoretic Techniques for Online Adaptation of Deep Neural Networks in Dynamical Systems

Observer-Side Parameter Estimation For Adaptive Control

Indirect Adaptive Control of Nonlinearly Parameterized Nonlinear Dissipative Systems

Online Optimization of Dynamical Systems with Deep Learning Perception

Neural Adaptive Control for Uncertain Mimo Systems with Constrained Input Via Intercepted Adaptation and Single Learning Parameter Approach

Dynamic Learning from Adaptive Neural Network Control of a Class of Nonaffine Nonlinear Systems

Data-driven learning and control of nonlinear system dynamics