Abstract:The localization of objects is a crucial task in various applications such as robotics, virtual and augmented reality, and the transportation of goods in warehouses. Recent advances in deep learning have enabled the localization using monocular visual cameras. While structure from motion (SfM) predicts the absolute pose from a point cloud, absolute pose regression (APR) methods learn a semantic understanding of the environment through neural networks. However, both fields face challenges caused by the environment such as motion blur, lighting changes, repetitive patterns, and feature-less structures. This study aims to address these challenges by incorporating additional information and regularizing the absolute pose using relative pose regression (RPR) methods. RPR methods suffer under different challenges, i.e., motion blur. The optical flow between consecutive images is computed using the Lucas-Kanade algorithm, and the relative pose is predicted using an auxiliary small recurrent convolutional network. The fusion of absolute and relative poses is a complex task due to the mismatch between the global and local coordinate systems. State-of-the-art methods fusing absolute and relative poses use pose graph optimization (PGO) to regularize the absolute pose predictions using relative poses. In this work, we propose recurrent fusion networks to optimally align absolute and relative pose predictions to improve the absolute pose prediction. We evaluate eight different recurrent units and construct a simulation environment to pre-train the APR and RPR networks for better generalized training. Additionally, we record a large database of different scenarios in a challenging large-scale indoor environment that mimics a warehouse with transportation robots. We conduct hyperparameter searches and experiments to show the effectiveness of our recurrent fusion method compared to PGO.

ViPR: Visual-Odometry-aided Pose Regression for 6DoF Camera Localization

CodeVIO: Visual-Inertial Odometry with Learned Optimizable Dense Depth

DF-VO: What Should Be Learnt for Visual Odometry?

PVO: Panoptic Visual Odometry.

Brain-Inspired Visual Odometry: Balancing Speed and Interpretability through a System of Systems Approach

Benchmarking Visual-Inertial Deep Multimodal Fusion for Relative Pose Regression and Odometry-aided Absolute Pose Regression

RNIN-VIO: Robust Neural Inertial Navigation Aided Visual-Inertial Odometry in Challenging Scenes

Visual Odometry with Deep Bidirectional Recurrent Neural Networks.

Leveraging Deep Learning for Visual Odometry Using Optical Flow

Deep Visual Odometry with Adaptive Memory

Salient Sparse Visual Odometry With Pose-Only Supervision

4DRVO-Net: Deep 4D Radar-Visual Odometry Using Multi-Modal and Multi-Scale Adaptive Fusion

Spatio-temporal and geometry constrained network for automobile visual odometry

Deep Visual Odometry with Events and Frames

Modality-invariant Visual Odometry for Embodied Vision

NVINS: Robust Visual Inertial Navigation Fused with NeRF-augmented Camera Pose Regressor and Uncertainty Quantification

NeRF-VO: Real-Time Sparse Visual Odometry with Neural Radiance Fields

Fusing Structure from Motion and Simulation-Augmented Pose Regression from Optical Flow for Challenging Indoor Environments

VOLDOR: Visual Odometry From Log-Logistic Dense Optical Flow Residuals

DeepAVO: Efficient Pose Refining with Feature Distilling for Deep Visual Odometry

Deep Direct Visual Odometry