Adaptive VIO: Deep Visual-Inertial Odometry with Online Continual Learning

Youqi Pan,Wugen Zhou,Yingdian Cao,Hongbin Zha

DOI: https://doi.org/10.48550/arXiv.2405.16754

2024-05-27

Abstract:Visual-inertial odometry (VIO) has demonstrated remarkable success due to its low-cost and complementary sensors. However, existing VIO methods lack the generalization ability to adjust to different environments and sensor attributes. In this paper, we propose Adaptive VIO, a new monocular visual-inertial odometry that combines online continual learning with traditional nonlinear optimization. Adaptive VIO comprises two networks to predict visual correspondence and IMU bias. Unlike end-to-end approaches that use networks to fuse the features from two modalities (camera and IMU) and predict poses directly, we combine neural networks with visual-inertial bundle adjustment in our VIO system. The optimized estimates will be fed back to the visual and IMU bias networks, refining the networks in a self-supervised manner. Such a learning-optimization-combined framework and feedback mechanism enable the system to perform online continual learning. Experiments demonstrate that our Adaptive VIO manifests adaptive capability on EuRoC and TUM-VI datasets. The overall performance exceeds the currently known learning-based VIO methods and is comparable to the state-of-the-art optimization-based methods.

Robotics

What problem does this paper attempt to address?

The paper aims to address the issue of insufficient generalization capability of Visual-Inertial Odometry (VIO) under different environments and sensor properties. Specifically, existing VIO methods perform poorly in challenging scenarios such as low-light conditions and sudden movements, mainly due to reliance on low-level handcrafted feature matching and inaccurate modeling of IMU biases. To solve these problems, the paper proposes a new system called Adaptive VIO, which combines online continual learning with traditional nonlinear optimization techniques.

The main features of Adaptive VIO include:
- **Combining Learning and Optimization**: Predicting visual correspondences and IMU biases through neural networks, and using traditional nonlinear optimization for state estimation.
- **Feedback Mechanism**: Using refined estimates obtained from optimization as feedback information to generate loss functions for updating the network, achieving self-supervised learning.
- **Online Continual Learning**: Continuously adjusting network parameters to enable the system to adapt to different environments and achieve better tracking performance in complex scenarios.

Experimental results show that Adaptive VIO demonstrates strong generalization and adaptability on the EuRoC and TUM-VI datasets, outperforming currently known learning-based VIO methods and being comparable to state-of-the-art optimization-based methods.