Event-based Visual Inertial Velometer

Xiuyuan Lu,Yi Zhou,Junkai Niu,Sheng Zhong,Shaojie Shen

2024-05-31

Abstract:Neuromorphic event-based cameras are bio-inspired visual sensors with asynchronous pixels and extremely high temporal resolution. Such favorable properties make them an excellent choice for solving state estimation tasks under aggressive ego motion. However, failures of camera pose tracking are frequently witnessed in state-of-the-art event-based visual odometry systems when the local map cannot be updated in time. One of the biggest roadblocks for this specific field is the absence of efficient and robust methods for data association without imposing any assumption on the environment. This problem seems, however, unlikely to be addressed as in standard vision due to the motion-dependent observability of event data. Therefore, we propose a mapping-free design for event-based visual-inertial state estimation in this paper. Instead of estimating the position of the event camera, we find that recovering the instantaneous linear velocity is more consistent with the differential working principle of event cameras. The proposed event-based visual-inertial velometer leverages a continuous-time formulation that incrementally fuses the heterogeneous measurements from a stereo event camera and an inertial measurement unit. Experiments on the synthetic dataset demonstrate that the proposed method can recover instantaneous linear velocity in metric scale with low latency.

Robotics

What problem does this paper attempt to address?

The paper primarily aims to address the challenges encountered when using event cameras for visual-inertial state estimation, particularly in high-dynamic motion scenarios such as high-speed drone flights. Specifically, the paper attempts to solve the following key issues:

1. **Data Association Problem**: Existing methods struggle to efficiently and robustly achieve data association without assuming environmental characteristics, especially when event data is motion-dependent.

2. **Map Update Problem**: When the local map cannot be updated in time, the camera pose tracking in existing event-based visual odometry systems frequently fails.

3. **Limitations of Fusion Methods**: Although some studies have attempted to address the above issues through visual-inertial fusion or feature-based methods, these approaches are either limited by specific environmental conditions or rely on complex image processing techniques, which restrict their applicability and real-time performance in high-dynamic scenarios.

To address these problems, the paper proposes a new design approach—an event-based visual-inertial velometer that does not require a map. This method focuses on recovering instantaneous linear velocity from the data of the event camera and inertial measurement unit (IMU), rather than directly estimating the camera's position. The main contributions of the paper include:

- **Novel State Estimator**: A novel method is proposed that utilizes the differential working principle of the event camera and fuses event data with IMU measurements to recover first-order dynamic states (i.e., linear velocity).

- **Normal Flow Computation**: A rigorous method is derived for computing normal flow from the spatiotemporal gradients of event data.

- **Continuous-Time Framework**: A continuous-time framework is developed for event-based visual-inertial fusion, capable of handling asynchronous event measurements and establishing data association between temporally misaligned acceleration measurements.

Experimental results show that the proposed method can effectively recover instantaneous linear velocity in metric scale on both synthetic and real data, with low latency characteristics. Additionally, the paper discusses initialization methods and system implementation details, providing qualitative and quantitative evaluations that demonstrate advantages over alternative solutions.