Abstract:Two problems exist in traditional multi-view geometry method to obtain the three-dimensional structure of the scene. First, the mismatching of the feature points caused by the blurred image and low texture, which reduces the accuracy of reconstruction; second, as the information obtained by monocular camera is lack of scale, the reconstruction results can only determine the unknown scale factor, and cannot get accurate scene structure. This paper proposes a method of equal-scale motion restoration structure based on deep learning. First, the convolutional neural network is used to obtain the depth information of the image; then, to restore the scale information of the monocular camera, an inertial measurement unit (IMU) is introduced, and the acceleration and angular velocity acquired by the IMU and the camera position acquired by the ORB-SLAM2 are demonstrated. The pose is coordinated in both time domain and frequency domain, and the scale information from the monocular camera is acquired in the frequency domain; finally, the depth information of the image and the camera pose with the scale factor are merged to reconstruct the three-dimensional structure of the scene. Experiments show that the monocular image depth map obtained by the Depth CNN network solves the problem that the output image of the multi-level convolution pooling operation has low resolution and lacks important feature information, and the absolute value error reaches 0.192, and the accuracy rate is up to 0.959. The multi-sensor fusion method can achieve a scale error of 0.24 m in the frequency domain, which is more accurate than that of the VIORB method in the frequency domain. The error between the reconstructed 3D model and the real size is about 0.2 m, which verifies the effectiveness of the proposed method.

Self-Supervised Scale Recovery for Monocular Depth and Egomotion Estimation

Unsupervised Learning of Depth and Ego-Motion with Absolutely Global Scale Recovery from Visual and Inertial Data Sequences

Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video

Monocular Depth Estimation Based on Multi-Scale Graph Convolution Networks

SelfOdom: Self-supervised Egomotion and Depth Learning via Bi-directional Coarse-to-Fine Scale Recovery

Accurate and Robust Scale Recovery for Monocular Visual Odometry Based on Plane Geometry

Towards Scale-Aware, Robust, and Generalizable Unsupervised Monocular Depth Estimation by Integrating IMU Motion Dynamics

A self‐supervised monocular depth estimation model with scale recovery and transfer learning for construction scene analysis

SelfTune: Metrically Scaled Monocular Depth Estimation through Self-Supervised Learning

Monocular Depth and Ego-motion Estimation with Scale Based on Superpixel and Normal Constraints

Unsupervised Scale-Consistent Depth Learning from Video

Self-supervised Monocular Depth and Visual Odometry Learning with Scale-consistent Geometric Constraints

Towards Scale-Aware Self-Supervised Multi-Frame Depth Estimation with IMU Motion Dynamics.

Marker-Less 3d Human Motion Capture With Monocular Image Sequence And Height-Maps

Self-Supervised Learning of Depth and Ego-Motion from Videos by Alternative Training and Geometric Constraints from 3-D to 2-D

Scale Recovery Using Vanishing Points for Indoor Mobile Vehicles With Monocular Vision

AVS-Net: Audio-Visual Scale Net for Self-supervised Monocular Metric Depth Estimation

Learning to Recover 3D Scene Shape from a Single Image

RA-Depth: Resolution Adaptive Self-Supervised Monocular Depth Estimation

GroCo: Ground Constraint for Metric Self-Supervised Monocular Depth

Equal-scale Structure from Motion Method Based on Deep Learning