Abstract:Two problems exist in traditional multi-view geometry method to obtain the three-dimensional structure of the scene. First, the mismatching of the feature points caused by the blurred image and low texture, which reduces the accuracy of reconstruction; second, as the information obtained by monocular camera is lack of scale, the reconstruction results can only determine the unknown scale factor, and cannot get accurate scene structure. This paper proposes a method of equal-scale motion restoration structure based on deep learning. First, the convolutional neural network is used to obtain the depth information of the image; then, to restore the scale information of the monocular camera, an inertial measurement unit (IMU) is introduced, and the acceleration and angular velocity acquired by the IMU and the camera position acquired by the ORB-SLAM2 are demonstrated. The pose is coordinated in both time domain and frequency domain, and the scale information from the monocular camera is acquired in the frequency domain; finally, the depth information of the image and the camera pose with the scale factor are merged to reconstruct the three-dimensional structure of the scene. Experiments show that the monocular image depth map obtained by the Depth CNN network solves the problem that the output image of the multi-level convolution pooling operation has low resolution and lacks important feature information, and the absolute value error reaches 0.192, and the accuracy rate is up to 0.959. The multi-sensor fusion method can achieve a scale error of 0.24 m in the frequency domain, which is more accurate than that of the VIORB method in the frequency domain. The error between the reconstructed 3D model and the real size is about 0.2 m, which verifies the effectiveness of the proposed method.

Camera Tracking with Constraint

Stereo Camera Motion Estimation Based on Euclidean Reconstruction

Camera Calibration Technique Based on Virtual 3D Object

Developing Rigid Constraint for the Estimation of Pose and Structure from a Single Image

Camera Tracking Based on Structure and Motion Recovery for Augmented Video

TC-SfM: Robust Track-Community-Based Structure-from-Motion

Two-Stage Multi-Camera Constrain Mapping Pipeline for Large-Scale 3D Reconstruction

Robust Visual Tracking Via CAMShift and Structural Local Sparse Appearance Model

Vid2Curve: simultaneous camera motion estimation and thin structure reconstruction from an RGB video

Efficient Non-Consecutive Feature Tracking for Robust Structure-From-Motion

Robust Camera Motion Estimation in Video Sequences

Efficient Non-Consecutive Feature Tracking For Structure-From-Motion

Recovering 3D Planar Arrangements from Videos

Monitoring activities from multiple video streams: establishing a common coordinate frame

Keyframe-based real-time camera tracking

Constrained Bundle Adjustment for Structure From Motion Using Uncalibrated Multi-Camera Systems

Vid2Curve

Robust Plane-Based Structure From Motion

Vision-based automatic structural motion estimation in presence of 3D camera motion effects

Equal-scale Structure from Motion Method Based on Deep Learning

Displacement measurement and nonlinear structural system identification: A vision‐based approach with camera motion correction using planar structures