Abstract:Omnidirectional multi-view stereo (MVS) vision is attractive for its ultra-wide field-of-view (FoV), enabling machines to perceive 360°3D surroundings. However, the existing solutions require expensive dense depth labels for supervision, making them impractical in real-world applications. In this paper, we propose the first unsupervised omnidirectional MVS framework based on multiple fisheye images. To this end, we project all images to a virtual view center and composite two panoramic images with spherical geometry from two pairs of back-to-back fisheye images. The two 360° images formulate a stereo pair with a special pose, and the photometric consistency is leveraged to establish the unsupervised constraint, which we term “Pseudo-Stereo Supervision”. In addition, we propose Un-OmniMVS, an efficient unsupervised omnidirectional MVS network, to facilitate the inference speed with two efficient components. First, a novel feature extractor with frequency attention is proposed to simultaneously capture the non-local Fourier features and local spatial features, explicitly facilitating the feature representation. Then, a variance-based light cost volume is put forward to reduce the computational complexity. Experiments exhibit that the performance of our unsupervised solution is competitive to that of the state-of-the-art (SoTA) supervised methods with better generalization in real-world data. The code will be available at https://github.com/Chen-z-s/Un-OmniMVS.

Self-Constructing Stereo Correspondences for Unsupervised Multi-View Stereo

Self-supervised Multi-view Stereo Via Inter and Intra Network Pseudo Depth

Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency

MVS2: Deep Unsupervised Multi-View Stereo with Multi-View Symmetry

MVS^2: Deep Unsupervised Multi-view Stereo with Multi-View Symmetry

Self-supervised Multi-view Stereo Via Effective Co-Segmentation and Data-Augmentation.

Self-supervised Multi-view Stereo Via View Synthesis Representation Consistency

A contrastive learning based unsupervised multi-view stereo with multi-stage self-training strategy

Unsupervised multi-view stereo network based on multi-stage depth estimation

Digging into Uncertainty in Self-supervised Multi-view Stereo

A Multi-Task Network for Multi-View Stereo Reconstruction: when Semantic Consistency Based Clustering Meets Depth Estimation Optimization

Focusing on Cross Views Improves Reconstruction in Unsupervised Multi-View Stereo

RC-MVSNet: Unsupervised Multi-View Stereo with Neural Rendering

A Multitask Network for Multiview Stereo Reconstruction: When Semantic Consistency-Based Clustering Meets Depth Estimation Optimization

Geometric Prior-Guided Self-Supervised Learning for Multi-View Stereo

Two-stage Self-supervised MVS Network Using Adaptive Depth Sampling.

Multi-View Stereo Representation Revist: Region-Aware MVSNet

Stereo Matching by Self-supervision of Multiscopic Vision.

Self-Supervised Multi-view Stereo Via Adjacent Geometry Guided Volume Completion

CL-MVSNet: Unsupervised Multi-View Stereo with Dual-Level Contrastive Learning

Unsupervised OmniMVS: Efficient Omnidirectional Depth Inference Via Establishing Pseudo-Stereo Supervision