Abstract:Real-time dense 3-D reconstruction is one of the major challenges in computer vision and robotics. In this article, we propose a real-time 3-D reconstruction model with metric-scale, including a direct visual-inertial odometry with stereo cameras and a deep multiview stereo network. Aiming at the scale uncertainty of dense map constructed by monocular camera, we designed a direct stereo visual-inertial odometry (DSVIO). The odometry combines static stereo optimization with direct visual-inertial odometry, using left-right images to initialize the depth of feature points, which can significantly improve the accuracy of 6-degree of freedom (DoF) pose and metric scale in the active window. In the aspect of depth estimation, the minimizing photometric re-projection loss (MPRP) proposed by us can integrate the common viewpoints for depth estimation under different view to improve the performance of deep multiview stereo network (CVA-MVSNet). Finally, the predicted depth map is fused into the truncated signed distance function (TSDF) voxel volume. The experiment shows that the pose estimation of our visual odometer has state-of-the-art (SOTA) performance when the trajectory is smooth and low jitter. In the case of fast jitter, our method is still superior to the monocular visual-internal odometry of oriented fast and rotated brief-simultaneous localization and mapping3 (ORB-SLAM3), but slightly inferior to the stereo visual-internal odometry of ORB-SLAM3. In the experiment of depth estimation, MPRP effectively improves the performance of CVA-MVSNet, and all evaluation indicators were superior to the original method. Moreover, our method had good performance in real-time 3-D reconstruction with metric scale.

Multi-camera Stereo Vision Based on Weights

Weighted Multi-Camera Stereo Vision Based on Geometric Relationships of Feature Edges

A Simple Rectification Method for Linear Multi-Baseline Stereovision System

A New Method for Increasing Accuracy of Distance Measurement Based on Single Visual Camera

Stereo Calibration and Rectification for Omnidirectional Multi-Camera Systems

Measurement based on geometry and feature shape by weighted multi-view stereo.

Real-time Stereo Vision System Using Adaptive Weight Cost Aggregation Approach

Unconstrained Self-Calibration of Stereo Camera on Visually Impaired Assistance Devices.

Weighted method for 3D-point reconstruction in multi-vision system

High-Quality Depth Recovery Via Interactive Multi-view Stereo

Optimization Design of Camera Pose for Network Base on Multi-view Stereo Vision

Real-Time Dense Construction with Deep Multiview Stereo Using Camera and IMU Sensors

The 3D Reconstruction of Face Model with Active Structured Light and Stereo Vision Fusion

An Improved Stereo Matching Algorithm Based on Joint Similarity Measure and Adaptive Weights

Advances in Three-Dimensional Imaging Technologies Based on Single-Camera Stereo Vision

Improved camera calibration method based on perpendicularity compensation for binocular stereo vision measurement system.

A Multi-View 3D Reconstruction Method Based on Pixel Discretization Model for Robot Pose Tracking

A Stable, Efficient, and High-Precision Non-Coplanar Calibration Method: Applied for Multi-Camera-Based Stereo Vision Measurements

Multi-View Stereo Via Geometric Expansion And Depth Refinement

A Multi-Camera Calibration Technique Based on Active Vision

Stereo Matching Based on Two Cameras and One 3D Image Sensor.