Abstract:Recent works have combined monocular event camera and inertial measurement unit to estimate the $SE(3)$ trajectory. However, the asynchronicity of event cameras brings a great challenge to conventional fusion algorithms. In this paper, we present an asynchronous event-inertial odometry under a unified Gaussian Process (GP) regression framework to naturally fuse asynchronous data associations and inertial measurements. A GP latent variable model is leveraged to build data-driven motion prior and acquire the analytical integration capacity. Then, asynchronous event-based feature associations and integral pseudo measurements are tightly coupled using the same GP framework. Subsequently, this fusion estimation problem is solved by underlying factor graph in a sliding-window manner. With consideration of sparsity, those historical states are marginalized orderly. A twin system is also designed for comparison, where the traditional inertial preintegration scheme is embedded in the GP-based framework to replace the GP latent variable model. Evaluations on public event-inertial datasets demonstrate the validity of both systems. Comparison experiments show competitive precision compared to the state-of-the-art synchronous scheme.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively handle asynchronous data association and high - time - resolution observations in the fusion of asynchronous event cameras and inertial measurement units (IMUs). Specifically, the paper proposes an asynchronous event - inertial odometry system based on the unified Gaussian Process (GP) regression framework, aiming to naturally fuse asynchronous data association and inertial measurements, thereby improving the accuracy and robustness of robot state estimation. ### Main Contributions 1. **Asynchronous Data Fusion**: A method based on the GP regression framework is proposed, which can effectively handle the fusion problem of asynchronous event camera and IMU data. 2. **High - Time - Resolution**: By introducing the GP latent variable model, a data - driven motion prior is constructed, and the analytic integration ability is obtained to maintain high - time - resolution. 3. **Sliding - Window Optimization**: The sliding - window method is used for optimization at the back - end, and the computational efficiency is maintained by marginalizing historical states. 4. **Dual - System Comparison**: A dual - system is designed, in which one system uses the traditional IMU pre - integration scheme, and the other system uses the GP latent variable model, so as to conduct comparative experiments under the same visual front - end and GP framework. ### Method Overview - **Front - End**: The event - driven feature detection and tracking method (such as EKLT) is adopted to construct a fully asynchronous data association and maintain the high - time - resolution of the event camera. - **Back - End**: The GP regression framework is used to construct the IMU - induced GP and asynchronous data association into the factor graph simultaneously, and optimization is carried out by the sliding - window method. - **Marginalization Strategy**: A marginalization strategy is proposed to remove historical states and maintain the consistency and sparsity of the factor graph. ### Experimental Results - **Accuracy**: Evaluations were carried out on multiple public event - inertial data sets, and the results show that the performance of this method in complex motion scenarios is comparable to that of existing methods, and even better in some sequences. - **Time Consumption**: Although GP - IMU pre - integration is slightly slower than standard IMU pre - integration, the overall graph optimization time is lower, and the computational cost is lower. ### Conclusion The asynchronous event - inertial odometry system based on the GP regression framework proposed in the paper performs excellently in complex motion scenarios and has high accuracy and robustness. Future work will focus on further optimizing the real - time performance and robustness of the system, for example, sparsifying the measurement factors through information theory.

Asynchronous Event-Inertial Odometry using a Unified Gaussian Process Regression Framework

AsynEIO: Asynchronous Monocular Event-Inertial Odometry Using Gaussian Process Regression

Intensity/Inertial Integration-Aided Feature Tracking on Event Cameras

AsynEVO: Asynchronous Event-Driven Visual Odometry for Pure Event Streams

Gyro-aided Camera-Odometer Online Calibration and Localization

GP-SLAM plus : real-time 3D lidar SLAM based on improved regionalized Gaussian process map reconstruction

Asynchronous Optimisation for Event-based Visual Odometry.

Asynchronous Event Feature Generation and Tracking Based on Gradient Descriptor for Event Cameras

Event-Based Visual Odometry on Non-Holonomic Ground Vehicles

Event-based Stereo Visual Odometry with Native Temporal Resolution via Continuous-time Gaussian Process Regression

Event-based Visual Inertial Velometer

Cascaded Motion Estimation for Intelligent Vehicles in GNSS-Denied Scenes

Traj-LIO: A Resilient Multi-LiDAR Multi-IMU State Estimator Through Sparse Gaussian Process

Inertial-aided State and Slope Estimation Using a Monocular Camera

Event-based Sensor Fusion and Application on Odometry: A Survey

Multisensor Estimation Fusion with Gaussian Process for Nonlinear Dynamic Systems

Tight Fusion of Events and Inertial Measurements for Direct Velocity Estimation

Continuous-Time Gaussian Process Motion-Compensation for Event-vision Pattern Tracking with Distance Fields

Stereo Event-based Visual-Inertial Odometry

Fusion of Inertial and Vision Data for Accurate Tracking

IncEventGS: Pose-Free Gaussian Splatting from a Single Event Camera