Abstract:Depth estimation attracts widespread attention in the computer vision community. However, it is still quite difficult to recover an accurate depth map using only one RGB image. We observe a phenomenon that existing methods tend to fail in different cases, caused by differences in network architecture, loss function and so on. In this work, we investigate into the phenomenon and propose to integrate the strengths of multiple weak depth predictor to build a comprehensive and accurate depth predictor, which is critical for many real-world applications, e.g., 3D reconstruction. Specifically, we construct multiple base (weak) depth predictors by utilizing different Transformer-based and convolutional neural network (CNN)-based architectures. Transformer establishes long-range correlation while CNN preserves local information ignored by Transformer due to the spatial inductive bias. Therefore, the coupling of Transformer and CNN contributes to the generation of complementary depth estimates, which are essential to achieve a comprehensive depth predictor. Then, we design mixers to learn from multiple weak predictions and adaptively fuse them into a strong depth estimate. The resultant model, which we refer to as Transformer-assisted depth ensembles (TEDepth). On the standard NYU-Depth-v2 and KITTI datasets, we thoroughly explore how the neural ensembles affect the depth estimation and demonstrate that our TEDepth achieves better results than previous state-of-the-art approaches. To validate the generalizability across cameras, we directly apply the models trained on NYU-Depth-v2 to the SUN RGB-D dataset without any fine-tuning, and the superior results emphasize its strong generalizability. The source code and trained models will be publicly available upon the acceptance.

Robust Depth Estimation Based on Parallax Attention for Aerial Scene Perception

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

A Robust Monocular Depth Estimation Framework Based on Light-Weight ERF-Pspnet for Day-Night Driving Scenes

Parallax attention stereo matching network based on the improved group-wise correlation stereo network

Expanding Sparse LiDAR Depth and Guiding Stereo Matching for Robust Dense Depth Estimation

DepthFormer: Exploiting Long-range Correlation and Local Information for Accurate Monocular Depth Estimation

Attention-Based Monocular Depth Estimation Considering Global and Local Information in Remote Sensing Images

Boosting Monocular Depth Estimation with Sparse Guided Points

Dense monocular depth estimation for stereoscopic vision based on pyramid transformer and multi-scale feature fusion

Platelet release products modulate some aspects of polymorphonuclear leukocyte activation

Depth-aware Volume Attention for Texture-less Stereo Matching

Robust and accurate depth estimation by fusing LiDAR and Stereo

Towards Comprehensive Monocular Depth Estimation: Multiple Heads are Better Than One

Unveiling the Depths: A Multi-Modal Fusion Framework for Challenging Scenarios

Depth Estimation Matters Most: Improving Per-Object Depth Estimation for Monocular 3D Detection and Tracking

SPDET: Edge-Aware Self-Supervised Panoramic Depth Estimation Transformer With Spherical Geometry

Self-Supervised Monocular Depth Estimation Based on High-Order Spatial Interactions

ClearDepth: Enhanced Stereo Perception of Transparent Objects for Robotic Manipulation

DELTAS: Depth Estimation by Learning Triangulation And densification of Sparse points

Monocular depth estimation via cross-spectral stereo information fusion

Parallax Attention for Unsupervised Stereo Correspondence Learning