Abstract:Our novel multi‐view stereo network, assisted by monocular prediction, addresses limitations in weak and repetitive texture regions. Combining monocular and multi‐view branches, we leverage semantic information from the single image and geometric relationships among multiple images. A coarse‐to‐fine strategy optimises computational efficiency while maintaining effectiveness. Our method achieves outstanding results, particularly in textureless regions, as demonstrated through experiments on DTU, Tanks and Temples, and BlendedMVS datasets. The learning‐based multi‐view stereo (MVS) methods have made remarkable progress in recent years. However, these methods exhibit limited robustness when faced with occlusion, weak or repetitive texture regions in the image. These factors often lead to holes in the final point cloud model due to excessive pixel‐matching errors. To address these challenges, we propose a novel MVS network assisted by monocular prediction for 3D reconstruction. Our approach combines the strengths of both monocular and multi‐view branches, leveraging the internal semantic information extracted from a single image through monocular prediction, along with the strict geometric relationships between multiple images. Moreover, we adopt a coarse‐to‐fine strategy to gradually reduce the number of assumed depth planes and minimise the interval between them as the resolution of the input images increases during the network iteration. This strategy can achieve a balance between the computational resource consumption and the effectiveness of the model. Experiments on the DTU, Tanks and Temples, and BlendedMVS datasets demonstrate that our method achieves outstanding results, particularly in textureless regions.

Adaptive region aggregation for multi‐view stereo matching using deformable convolutional networks

N2MVSNet: Non-Local Neighbors Aware Multi-View Stereo Network

MFE‐MVSNet: Multi‐scale feature enhancement multi‐view stereo with bi‐directional connections

CNLPA-MVS: Coarse-Hypotheses Guided Non-Local PatchMatch Multi-View Stereo

DSC-MVSNet: attention aware cost volume regularization based on depthwise separable convolution for multi-view stereo

AA-RMVSNet: Adaptive Aggregation Recurrent Multi-view Stereo Network

Multi-view depth estimation based on multi-feature aggregation for 3D reconstruction

Multi-View Stereo Representation Revist: Region-Aware MVSNet

HC-MVSNet: A Probability Sampling-Based Multi-View-stereo Network with Hybrid Cascade Structure for 3D Reconstruction

Adaptive Cost Aggregation in Iterative Depth Estimation for Efficient Multi-view Stereo.

Adaptive deformable convolutional network

Multi-View Stereo Network Based on Attention Mechanism and Neural Volume Rendering

Self-adaptive Multi-scale Aggregation Network for Stereo Matching.

MTD-MVSNet: Multi-view Stereo Network with Multi-scale Transformer and Dual Attention

NR-MVSNet: Learning Multi-View Stereo Based on Normal Consistency and Depth Refinement

Adaptive Patch Deformation for Textureless-Resilient Multi-View Stereo

Enhanced feature pyramid for multi-view stereo with adaptive correlation cost volume

FA-MSVNet: multi-scale and multi-view feature aggregation methods for stereo 3D reconstruction

OD-MVSNet: Omni-dimensional dynamic multi-view stereo network

EI-MVSNet: Epipolar-Guided Multi-View Stereo Network With Interval-Aware Label

Mono‐MVS: textureless‐aware multi‐view stereo assisted by monocular prediction