Abstract: Multi-view depth estimation plays a critical role in reconstructing and understanding the 3D world. Recent learning-based methods have made significant progress in it. However, multi-view depth estimation is fundamentally a correspondence-based optimization problem, but previous learning-based methods mainly rely on predefined depth hypotheses to build correspondence as the cost volume and implicitly regularize it to fit depth prediction, deviating from the essence of iterative optimization based on stereo correspondence. Thus, they suffer unsatisfactory precision and generalization capability. In this paper, we are the first to explore more general image correlations to establish correspondences dynamically for depth estimation. We design a novel iterative multi-view depth estimation framework mimicking the optimization process, which consists of 1) a correlation volume construction module that models the pixel similarity between a reference image and source images as all-to-all correlations; 2) a flow-based depth initialization module that estimates the depth from the 2D optical flow; 3) a novel correlation-guided depth refinement module that reprojects points in different views to effectively fetch relevant correlations for further fusion and integrate the fused correlation for iterative depth update. Without predefined depth hypotheses, the fused correlations establish multi-view correspondence in an efficient way and guide the depth refinement heuristically. We conduct sufficient experiments on ScanNet, DeMoN, ETH3D, and 7Scenes to demonstrate the superiority of our method on multi-view depth estimation and its best generalization ability.

A LBP Based Correspondence Identification Scheme for Multi-view Sensing Network

2-Entity Random Sample Consensus for Robust Visual Localization: Framework, Methods, and Verifications

Leveraging Local Planar Motion Property for Robust Visual Matching and Localization.

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Neural Best-Buddies: Sparse Cross-Domain Correspondence

Robust 3D Reconstruction with an RGB-D Camera

(LC)$^2$: LiDAR-Camera Loop Constraints For Cross-Modal Place Recognition

Universal Correspondence Network

A novel correspondence searching strategy in multiocular vision

Visual Descriptor Learning from Monocular Video

Cycle-Correspondence Loss: Learning Dense View-Invariant Visual Features from Unlabeled and Unordered RGB Images

RKHS-BA: A Robust Correspondence-Free Multi-View Registration Framework with Semantic Point Clouds

Monocular Visual Place Recognition in LiDAR Maps via Cross-Modal State Space Model and Multi-View Matching

SCSA-Net: Presentation of two-view reliable correspondence learning via spatial-channel self-attention

Multi-view object pose estimation from correspondence distributions and epipolar geometry

Lpcv: Learning Projections From Corresponding Views For Person Re-Identification

A Branch-and-Bound Approach to Correspondence and Grouping Problems

Exploiting Correspondences with All-pairs Correlations for Multi-view Depth Estimation

BCLNet: Bilateral Consensus Learning for Two-View Correspondence Pruning

Efficient and Outlier-Robust Simultaneous Pose and Correspondence Determination by Branch-and-Bound and Transformation Decomposition

Learning 2D-3D Correspondences To Solve The Blind Perspective-n-Point Problem