NPF-MVSNet: Normal and Pyramid Feature Aided Unsupervised MVS Network.

Baichuan Huang,Hongwei Yi,Can Huang,Yijia He,Jingbin Liu,Xiao Liu

2020-01-01

Abstract:The present Multi-view stereo (MVS) methods with supervised learning-basednetworks have an impressive performance comparing with traditional MVS methods.However, the ground-truth depth maps for training are hard to be obtained andare within limited kinds of scenarios. In this paper, we propose a novelunsupervised multi-metric MVS network, named M^3VSNet, for dense point cloudreconstruction without any supervision. To improve the robustness andcompleteness of point cloud reconstruction, we propose a novel multi-metricloss function that combines pixel-wise and feature-wise loss function to learnthe inherent constraints from different perspectives of matchingcorrespondences. Besides, we also incorporate the normal-depth consistency inthe 3D point cloud format to improve the accuracy and continuity of theestimated depth maps. Experimental results show that M3VSNet establishes thestate-of-the-arts unsupervised method and achieves comparable performance withprevious supervised MVSNet on the DTU dataset and demonstrates the powerfulgeneralization ability on the Tanks and Temples benchmark with effectiveimprovement. Our code is available at https://github.com/whubaichuan/M3VSNet.