Abstract:Existing voxel-hashing (VH)-based dense reconstruction methods have shown impressive results on datasets collected by hand-held cameras. Large-scale scenes are maintained with a truncated signed distance function (TSDF) volumetric representation. However, practically deploying such methods on low-cost embedded mobile robots remains challenging due to heavy computational burdens and various camera perception degeneration cases. In this work, we propose FROMFusion, a fast and robust on-manifold dense reconstruction framework based on multisensor fusion, which systematically solves how to align the point cloud with the hashed TSDF volume (HTV). Its purely geometric nature ensures the robustness to image motion blur and poor lighting conditions. To reduce memory overhead, we propose a spherical-coordinate-based HTV segmentation algorithm. To surmount missing geometric features, camera occlusion, and over range, a loosely coupled LiDAR-wheel-inertial odometry (LWIO) is applied for trustworthy initial guesses in camera pose optimization. At its core is a two-stage depth-to-HTV matching algorithm, which includes a coarse voxel-level pose ergodic search and a fine subvoxel-level Gauss-Newton (GN) solver with Anderson acceleration (AA) strategy for faster convergence. We evenly distribute heavy computational workloads to heterogeneous computing systems. Extensive field experiments on a low-cost wheeled robot cleaner demonstrate our method models continuous surfaces of large-scale scenes with high quality in both geometry and texture, outperforming current state-of-the-art methods in robustness to camera perception degeneration cases by a significant margin. The frame rate of online embedded implementation can reach up to 47.21 Hz maximum.

Idfusion

FastFusion: Real-Time Indoor Scene Reconstruction with Fast Sensor Motion

LF-fusion: Dense and Accurate 3D Reconstruction from Light Field Images

DI-Fusion: Online Implicit 3D Reconstruction with Deep Priors

InstanceFusion: Real‐time Instance‐level 3D Reconstruction Using a Single RGBD Camera

3DFusion, A real-time 3D object reconstruction pipeline based on streamed instance segmented data

HeteroFusion: Dense Scene Reconstruction Integrating Multi-Sensors.

HybridFusion: Real-Time Performance Capture Using a Single Depth Sensor and Sparse IMUs

Online Global Non-rigid Registration for 3D Object Reconstruction Using Consumer-level Depth Cameras

RigidFusion: RGB‐D Scene Reconstruction with Rigidly‐moving Objects

FlashFusion: Real-time Globally Consistent Dense 3D Reconstruction Using CPU Computing.

FlyFusion: Realtime Dynamic Scene Reconstruction Using a Flying Depth Camera

UnstructuredFusion: Realtime 4D Geometry and Texture Reconstruction Using Commercial RGBD Cameras.

A New Volumetric Fusion Strategy with Adaptive Weight Field for RGB-D Reconstruction

Dense RGB-D-Inertial SLAM with Map Deformations

A Real-Time Fusion Framework for Long-term Visual Localization

FROMFusion: Fast and Robust On-Manifold Dense Reconstruction for Low-Cost Wheeled Robots

Intensity Video Guided 4D Fusion for Improved Highly Dynamic 3D Reconstruction

OwlFusion: Depth-Only Onboard Real-Time 3D Reconstruction of Scalable Scenes for Fast-Moving MAV

Real-time Indoor Scene Reconstruction with RGBD and Inertial Input.

DoubleFusion: Real-Time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor