Abstract:Research on multi-view stereo based on remote sensing images has promoted the development of large-scale urban 3D reconstruction. However, remote sensing multi-view image data suffers from the problems of occlusion and uneven brightness between views during acquisition, which leads to the problem of blurred details in depth estimation. To solve the above problem, we re-examine the deformable learning method in the Multi-View Stereo task and propose a novel paradigm based on view Space and Depth deformable Learning (SDL-MVS), aiming to learn deformable interactions of features in different view spaces and deformably model the depth ranges and intervals to enable high accurate depth estimation. Specifically, to solve the problem of view noise caused by occlusion and uneven brightness, we propose a Progressive Space deformable Sampling (PSS) mechanism, which performs deformable learning of sampling points in the 3D frustum space and the 2D image space in a progressive manner to embed source features to the reference feature adaptively. To further optimize the depth, we introduce Depth Hypothesis deformable Discretization (DHD), which achieves precise positioning of the depth prior by adaptively adjusting the depth range hypothesis and performing deformable discretization of the depth interval hypothesis. Finally, our SDL-MVS achieves explicit modeling of occlusion and uneven brightness faced in multi-view stereo through the deformable learning paradigm of view space and depth, achieving accurate multi-view depth estimation. Extensive experiments on LuoJia-MVS and WHU datasets show that our SDL-MVS reaches state-of-the-art performance. It is worth noting that our SDL-MVS achieves an MAE error of 0.086, an accuracy of 98.9% for <0.6m, and 98.9% for <3-interval on the LuoJia-MVS dataset under the premise of three views as input.

Hierarchical Prior Mining for Non-local Multi-View Stereo.

HC-MVSNet: A Probability Sampling-Based Multi-View-stereo Network with Hybrid Cascade Structure for 3D Reconstruction

MSP-MVS: Multi-granularity Segmentation Prior Guided Multi-View Stereo

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Geometric Prior-Guided Self-Supervised Learning for Multi-View Stereo

Sparse Prior Guided Deep Multi-View Stereo

LGP-MVS: Combined Local and Global Planar Priors Guidance for Indoor Multi-View Stereo

CNLPA-MVS: Coarse-Hypotheses Guided Non-Local PatchMatch Multi-View Stereo

HQP-MVS:High-Quality Plane Priors Assisted Multi-View Stereo for Low-Textured Areas

Multi-View Stereo Representation Revist: Region-Aware MVSNet

High completeness multi-view stereo for dense reconstruction of large-scale urban scenes

A Multitask Network for Multiview Stereo Reconstruction: When Semantic Consistency-Based Clustering Meets Depth Estimation Optimization

Unsupervised multi-view stereo network based on multi-stage depth estimation

MP-MVS: Multi-Scale Windows PatchMatch and Planar Prior Multi-View Stereo

N2MVSNet: Non-Local Neighbors Aware Multi-View Stereo Network

Self-supervised Multi-view Stereo Via Inter and Intra Network Pseudo Depth

EPP-MVSNet: Epipolar-assembling based Depth Prediction for Multi-view Stereo

SDL-MVS: View Space and Depth Deformable Learning Paradigm for Multi-View Stereo Reconstruction in Remote Sensing

GeoMVSNet: Learning Multi-View Stereo with Geometry Perception

NR-MVSNet: Learning Multi-View Stereo Based on Normal Consistency and Depth Refinement

Self-Supervised Multi-view Stereo Via Adjacent Geometry Guided Volume Completion