Abstract:In recent years, with the vigorous development of artificial intelligence and autonomous driving technology, the importance of scene perception technology is increasing. Unsupervised deep learning based methods have demonstrated a certain level of robustness and accuracy in some challenging scenes. By inferring depth from a single input image without any ground truth label, a lot of time and resources can be saved. However, unsupervised depth estimation has defects in robustness and accuracy under complex environment which could be improved by modifying network structure and incorporating other modal information. In this paper, we propose an unsupervised, monocular depth estimation network achieving high speed and accuracy, and a learning framework with our depth estimation network to improve depth performance by incorporating transformed images across different modalities. The depth estimator is an encoder-decoder network to generate the multi-scale dense depth map. The sub-pixel convolutional layer is adopted to obtain depth super-resolution by replacing the up-sample branches. The cross-modal depth estimation using near-infrared image and RGB image enhances the performance of depth estimation than pure RGB image. The training mode is to transfer both images to the same modality and then carry out super-resolved depth estimation for each stereo camera pair. Compared with the initial results of depth estimation using only RGB images, the experiment verifies that our depth estimation network with the cross-modal fusion system designed in this paper achieves better performance on public datasets and a multi-modal dataset collected by our stereo vision sensor.

REAL-TIME DEPTH MAP ESTIMATION FROM INFRARED STEREO IMAGES OF RGB-D CAMERAS

DoCam: Depth Sensing with an Optical Image Stabilization Supported RGB Camera.

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

IR Stereo RealSense: Decreasing Minimum Range of Navigational Assistance for Visually Impaired Individuals.

Improving RealSense by Fusing Color Stereo Vision and Infrared Stereo Vision for the Visually Impaired

The Real-Time Depth Map Obtainment Based on Stereo Matching.

Hardware Solution of Real-Time Depth Estimation Based on Stereo Vision

K-nearest Neighborhood Based Integration of Time-of-flight Cameras and Passive Stereo for High-Accuracy Depth Maps.

Accurate Depth Sensing by Filtering IR Cost on the Guide of RGB Information

Expanding Sparse LiDAR Depth and Guiding Stereo Matching for Robust Dense Depth Estimation

High-Quality Depth Recovery Via Interactive Multi-view Stereo

K-nearestneighborhood Basedintegrationoftime-of-flight Camerasandpassivestereoforhigh-Accuracydepthmaps ∗

DELTAR: Depth Estimation from a Light-Weight ToF Sensor and RGB Image

Real-time Depth Extraction and Multi-View Rendering Algorithm Based on Kinect

High quality depth maps from stereo matching and ToF camera.

Accurate Real-Time Stereo Correspondence Using Intra- and Inter-Scanline Optimization

A Self-Calibrated Photo-Geometric Depth Camera

Depth Estimation by Combining Binocular Stereo and Monocular Structured-Light

Real-Time Stereo Matching System

Real-Time Accurate Stereo Matching Using Modified Two-Pass Aggregation and Winner-Take-All Guided Dynamic Programming

A High Spatial Resolution Depth Sensing Method Based on Binocular Structured Light