Abstract:Fully perceiving the surrounding world is a vital capability for autonomous robots. To achieve this goal, a multi-camera system is usually equipped on the data collecting platform and the structure from motion (SfM) technology is used for scene reconstruction. However, although incremental SfM achieves high-precision modeling, it is inefficient and prone to scene drift in large-scale reconstruction tasks. In this paper, we propose a tailored incremental SfM framework for multi-camera systems, where the internal relative poses between cameras can not only be calibrated automatically but also serve as an additional constraint to improve the system robustness. Previous multi-camera based modeling work has mainly focused on stereo setups or multi-camera systems with known calibration information, but we allow arbitrary configurations and only require images as input. First, one camera is selected as the reference camera, and the other cameras in the multi-camera system are denoted as non-reference cameras. Based on the pose relationship between the reference and non-reference camera, the non-reference camera pose can be derived from the reference camera pose and internal relative poses. Then, a two-stage multi-camera based camera registration module is proposed, where the internal relative poses are computed first by local motion averaging, and then the rigid units are registered incrementally. Finally, a multi-camera based bundle adjustment is put forth to iteratively refine the reference camera and the internal relative poses. Experiments demonstrate that our system achieves higher accuracy and robustness on benchmark data compared to the state-of-the-art SfM and SLAM (simultaneous localization and mapping) methods.

Deep Non-rigid Structure-from-Motion Revisited: Canonicalization and Sequence Modeling

Deep Non-rigid Structure-from-Motion: A Sequence-to-Sequence Translation Perspective

Non-rigid Structure-from-Motion: Temporally-smooth Procrustean Alignment and Spatially-variant Deformation Modeling

Two-Stage Multi-Camera Constrain Mapping Pipeline for Large-Scale 3D Reconstruction

Structure from Recurrent Motion: from Rigidity to Recurrency

TC-SfM: Robust Track-Community-Based Structure-from-Motion

Monocular 3D Reconstruction of Multiple Non-Rigid Objects by Union of Non-linear Spatial-Temporal Subspaces.

Robust Isometric Non-Rigid Structure-From-Motion

DeepSFM: Robust Deep Iterative Refinement for Structure from Motion.

CRF-Based Reconstruction from Narrow-Baseline Image Sequences.

Deep Permutation Equivariant Structure from Motion

Recovering Complex Non-Rigid 3d Structures From Monocular Images By Union Of Nonlinear Subspaces

MCSfM: Multi-Camera-Based Incremental Structure-From-Motion

Visual Geometry Grounded Deep Structure From Motion

Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data

Level-S<SUP>2</SUP>fM: Structure from Motion on Neural Level Set of Implicit Surfaces

Level-S$^2$fM: Structure from Motion on Neural Level Set of Implicit Surfaces

DRO: Deep Recurrent Optimizer for Structure-from-Motion.

PR-RRN: Pairwise-Regularized Residual-Recursive Networks for Non-rigid Structure-from-Motion

Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling

DeepSFM: Structure from Motion via Deep Bundle Adjustment