Abstract:Real-time 3D reconstruction is one of the current popular research directions of computer vision, and it has become the core technology in the fields of virtual reality, industrialized automatic systems, and mobile robot path planning. Currently, there are three main problems in the real-time 3D reconstruction field. Firstly, it is expensive. It requires more varied sensors, so it is less convenient. Secondly, the reconstruction speed is slow, and the 3D model cannot be established accurately in real time. Thirdly, the reconstruction error is large, which cannot meet the requirements of scenes with accuracy. For this reason, we propose a real-time 3D reconstruction method based on monocular vision in this paper. Firstly, a single RGB-D camera is used to collect visual information in real time, and the YOLACT++ network is used to identify and segment the visual information to extract part of the important visual information. Secondly, we combine the three stages of depth recovery, depth optimization, and deep fusion to propose a three-dimensional position estimation method based on deep learning for joint coding of visual information. It can reduce the depth error caused by the depth measurement process, and the accurate 3D point values of the segmented image can be obtained directly. Finally, we propose a method based on the limited outlier adjustment of the cluster center distance to optimize the three-dimensional point values obtained above. It improves the real-time reconstruction accuracy and obtains the three-dimensional model of the object in real time. Experimental results show that this method only needs a single RGB-D camera, which is not only low cost and convenient to use, but also significantly improves the speed and accuracy of 3D reconstruction.

Real-Time 3D Reconstruction Using Time of Flight Sensor Optimized Visual Hull

Reliable Fusion of ToF and Stereo Data Based on Joint Depth Filter

Real-Time Artifact Compensation for Depth Images of Multi-Frequency ToF

Real-time Underwater 3D Reconstruction Based on Monocular Image

Visual Odometry Based 3D-Reconstruction

K-nearest Neighborhood Based Integration of Time-of-flight Cameras and Passive Stereo for High-Accuracy Depth Maps.

High quality depth maps from stereo matching and ToF camera.

Robust 3D Reconstruction with an RGB-D Camera

Complete 3D Model Reconstruction Using a Depth Sensor

DELTAR: Depth Estimation from a Light-Weight ToF Sensor and RGB Image

Accurate 3D reconstruction using a multi-phase ToF camera

Simultaneous Estimation of Object Region and Depth in Participating Media Using a ToF Camera

Real-time 3D Scene Reconstruction with Dynamically Moving Object Using a Single Depth Camera

3D Reconstruction Using 3D Registration-Based ToF-Stereo Fusion

A Robust and Real-Time Full 3D Reconstruction Method Based on Multiple Kinect

Improveing Depth Estimation Through Fusion of Stereo and TOF

FastFusion: Real-Time Indoor Scene Reconstruction with Fast Sensor Motion

Real-Time 3D Reconstruction Method Based on Monocular Vision

Real-time High-accuracy Three-Dimensional Reconstruction with Consumer RGB-D Cameras

Real-time Underwater 3D Reconstruction Method Based on Stereo Camera

Real-time high resolution 2D-plus-depth video generation using time-of-flight sensor