Abstract:Mobile AR applications benefit from fast initialization to display world-locked effects instantly. However, standard visual odometry or SLAM algorithms require motion parallax to initialize (see Figure 1) and, therefore, suffer from delayed initialization. In this paper, we present a 6-DoF monocular visual odometry that initializes instantly and without motion parallax. Our main contribution is a pose estimator that decouples estimating the 5-DoF relative rotation and translation direction from the 1-DoF translation magnitude. While scale is not observable in a monocular vision-only setting, it is still paramount to estimate a consistent scale over the whole trajectory (even if not physically accurate) to avoid AR effects moving erroneously along depth. In our approach, we leverage the fact that depth errors are not perceivable to the user during rotation-only motion. However, as the user starts translating the device, depth becomes perceivable and so does the capability to estimate consistent scale. Our proposed algorithm naturally transitions between these two modes. Our second contribution is a novel residual in the relative pose problem to further improve the results. The residual combines the Jacobians of the functional and the functional itself and is minimized using a Levenberg-Marquardt optimizer on the 5-DoF manifold. We perform extensive validations of our contributions with both a publicly available dataset and synthetic data. We show that the proposed pose estimator outperforms the classical approaches for 6-DoF pose estimation used in the literature in low-parallax configurations. Likewise, we show our relative pose estimator outperforms state-of-the-art approaches in an odometry pipeline configuration where we can leverage initial guesses. We release a dataset for the relative pose problem using real data to facilitate the comparison with future solutions for the relative pose problem. Our solution is either used as a full odometry or as a pre-SLAM component of any supported SLAM system (ARKit, ARCore) in world-locked AR effects on platforms such as Instagram and Facebook.

NEAR: the NetEase AR Oriented Visual Inertial Dataset

Survey and Evaluation of Monocular Visual-Inertial SLAM Algorithms for Augmented Reality

The Visual-Inertial- Dynamical Multirotor Dataset

ADVIO: An authentic dataset for visual-inertial odometry

The Visual-Inertial-Dynamical Multirotor Dataset

100-Phones: A Large VI-SLAM Dataset for Augmented Reality Towards Mass Deployment on Mobile Phones

RLP‐VIO: Robust and Lightweight Plane‐based Visual‐inertial Odometry for Augmented Reality

A Visual-Inertial Fusion Based Tracking System For Mobile Augmented Reality

RNIN-VIO: Robust Neural Inertial Navigation Aided Visual-Inertial Odometry in Challenging Scenes

InCrowd-VI: A Realistic Visual-Inertial Dataset for Evaluating SLAM in Indoor Pedestrian-Rich Spaces for Human Navigation

Amirkabir campus dataset: Real-world challenges and scenarios of Visual Inertial Odometry (VIO) for visually impaired people

Robust and Efficient Visual-Inertial Odometry with Multi-plane Priors.

OxIOD: The Dataset for Deep Inertial Odometry

RD-VIO: Robust Visual-Inertial Odometry for Mobile Augmented Reality in Dynamic Environments

The NEOLIX Open Dataset for Autonomous Driving

NTU VIRAL: A Visual-Inertial-Ranging-Lidar Dataset, From an Aerial Vehicle Viewpoint

A Benchmark Comparison of Four Off-the-Shelf Proprietary Visual-Inertial Odometry Systems

V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception

Instant Visual Odometry Initialization for Mobile AR

Road and Railway Smart Mobility: A High-Definition Ground Truth Hybrid Dataset

LiDAR-Net: A Real-scanned 3D Point Cloud Dataset for Indoor Scenes