Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model Predictive Control

Alessandro Saviolo,Jonathan Frey,Abhishek Rathod,Moritz Diehl,Giuseppe Loianno
DOI: https://doi.org/10.1109/TRO.2023.3339543
2024-08-31
Abstract:Model-based control requires an accurate model of the system dynamics for precisely and safely controlling the robot in complex and dynamic environments. Moreover, in the presence of variations in the operating conditions, the model should be continuously refined to compensate for dynamics changes. In this paper, we present a self-supervised learning approach that actively models the dynamics of nonlinear robotic systems. We combine offline learning from past experience and online learning from current robot interaction with the unknown environment. These two ingredients enable a highly sample-efficient and adaptive learning process, capable of accurately inferring model dynamics in real-time even in operating regimes that greatly differ from the training distribution. Moreover, we design an uncertainty-aware model predictive controller that is heuristically conditioned to the aleatoric (data) uncertainty of the learned dynamics. This controller actively chooses the optimal control actions that (i) optimize the control performance and (ii) improve the efficiency of online learning sample collection. We demonstrate the effectiveness of our method through a series of challenging real-world experiments using a quadrotor system. Our approach showcases high resilience and generalization capabilities by consistently adapting to unseen flight conditions, while it significantly outperforms classical and adaptive control baselines.
Robotics,Machine Learning,Systems and Control
What problem does this paper attempt to address?
The paper aims to address the problem of dynamic modeling in robotic control systems, particularly the issue of accurately and safely controlling robots in complex and dynamic environments. Specifically, the paper proposes a self-supervised learning method for actively modeling the discrete-time dynamics of nonlinear robots. This method combines offline learning from past experiences with online learning from the current interaction of the robot with unknown environments, making the learning process highly sample-efficient and adaptive. It can accurately infer the model dynamics under operational modes that differ significantly from the training distribution in real-time situations. Additionally, the paper designs an uncertainty-aware Model Predictive Controller (MPC) that can heuristically adjust based on the stochastic uncertainty (data uncertainty) of the learned dynamics. This controller can actively select the optimal control actions to optimize control performance and improve the efficiency of online learning sample collection. The effectiveness of this method is validated through a series of challenging real-world experiments using a quadrotor system. The method demonstrates the ability to continuously adapt to unseen flight conditions and significantly outperforms classical and adaptive control baseline methods.