Abstract:In recent years, self-supervised depth and ego- motion estimation have attracted extensive research attention. Self-supervised monocular depth estimation can effectively learn information from texture-less and non-Lambertian surfaces. However, the monocular methods' accuracy is limited owing to the depth ambiguity in the imaging principles of monocular cameras. On the other hand, multi-frame methods usually have higher depth accuracy benefitting from the geometric constraints of Multi-View Stereo (MVS). However, MVS is sensitive to non- Lambertian surfaces, moving objects, and texture-less regions, which often leads to wrong estimation. In addition, both methods are difficult to obtain the absolute scale result owing to the essential scale ambiguity of monocular images. To resolve the above problems, we propose the FusionDepth, a scale-aware method that fuses the information from monocular depth estimation, multi-frame depth estimation, and Inertial Measurement Unit (IMU) in this paper. FusionDepth fuses multi-frame depth estimation and monocular depth estimation through an uncertainty mask to realize complementary advantages. It further fuses IMU information with an Extended Kalman Filter (EKF) to learn an absolute scale metric. To run the model in real-time on resource-limited devices, we compact the framework by sharing the encoder and taking two lightweight decoders to lower the expense of pose prediction and the depth prediction of MVS. We verify FusionDepth's effectiveness on the KITTI benchmark and the comparison experiments indicate our method is efficient and accurate.

An adaptive motion estimation algorithm based on mutual information for depth information estimation

Bio-inspired Motion-Adaptive Estimation Algorithm of Sequence Image

A Depth Extraction Method Based on Motion and Geometry for 2D to 3D Conversion

A New Adaptive Motion Estimation Algorithm Based on Correlation of Motion Vectors

Macroblock-Level Adaptive Search Range Algorithm for Motion Estimation in Multiview Video Coding

Mutual information based non-parametric transform stereo matching algorithm

Inter Mode Selection for Depth Map Coding in 3D Video.

Selective Disparity Estimation and Variable Size Motion Estimation Based on Motion Homogeneity for Multi-View Coding

Depth Estimation Based on Adaptive Support Weight and SIFT for Multi-Lenslet Cameras

Cross-Hexagon-based Motion Estimation Algorithm Using Motion Vector Adaptive Search Technique

Fast Binocular Depth Inference via Bidirectional Motion Based Interpolation

An Adaptive Fast Video Encoding/decoding Scheme Based on Mutual Information

Video segmentation based motion estimation for stereoscopic video

Monocular Piecewise Depth Estimation in Dynamic Scenes by Exploiting Superpixel Relations

An Efficient 3D Video Frame Interpolation Method Using Color-Depth-motion Information

Motion Estimation for Depth Maps Coding Based on Rendered View Distortion

Inter-view motion vector prediction for depth coding

Method of 2.5-D inter-frame motion estimation

Adaptive Dynamic Search Range Algorithm for Motion Estimation

Towards Scale-Aware Self-Supervised Multi-Frame Depth Estimation with IMU Motion Dynamics.

Depth Map Estimation of Dynamic Scenes Using Prior Depth Information