Abstract:We present a novel framework named PlaneMVS for 3D plane reconstruction from multiple input views with known camera poses. Most previous learning-based plane reconstruction methods reconstruct 3D planes from single images, which highly rely on single-view regression and suffer from depth scale ambiguity. In contrast, we reconstruct 3D planes with a multi-view-stereo (MVS) pipeline that takes advantage of multi-view geometry. We decouple plane reconstruction into a semantic plane detection branch and a plane MVS branch. The semantic plane detection branch is based on a single-view plane detection framework but with differences. The plane MVS branch adopts a set of slanted plane hypotheses to replace conventional depth hypotheses to perform plane sweeping strategy and finally learns pixel-level plane parameters and its planar depth map. We present how the two branches are learned in a balanced way, and propose a soft-pooling loss to associate the outputs of the two branches and make them benefit from each other. Extensive experiments on various indoor datasets show that PlaneMVS significantly outperforms state-of-the-art (SOTA) single-view plane reconstruction methods on both plane detection and 3D geometry metrics. Our method even outperforms a set of SOTA learning-based MVS methods thanks to the learned plane priors. To the best of our knowledge, this is the first work on 3D plane reconstruction within an end-to-end MVS framework. Source code: <a class="link-external link-https" href="https://github.com/oppo-us-research/PlaneMVS" rel="external noopener nofollow">this https URL</a>.

Line-based single view 3D reconstruction in Manhattan world for augmented reality

Novel Single View Constraints for Manhattan 3D Line Reconstruction

An Example-Based Approach to 3D Man-Made Object Reconstruction from Line Drawings

Neural 3D Scene Reconstruction with the Manhattan-world Assumption

Progressive 3D Reconstruction of Planar-Faced Manifold Objects with DRF-Based Line Drawing Decomposition

Image-based building reconstruction with Manhattan-world assumption

A new geometric approach for three view line reconstruction and motion estimation in Manhattan Scenes

Learning to Reconstruct 3D Manhattan Wireframes from a Single Image

Line-Based 3D Building Abstraction and Polygonal Surface Reconstruction From Images

Outdoor Scene 3D Reconstruction from Multiple Point Cloud

Multi-view Urban Scene Reconstruction in Non-Uniform Volume

3-D Reconstruction of Three Views Based on Manifold Study

Neural 3D Scene Reconstruction with Indoor Planar Priors

DP-MVS: Detail Preserving Multi-View Surface Reconstruction of Large-Scale Scenes

Multi-View Depth Map Sampling for 3D Reconstruction of Natural Scene

Real-time indoor scene reconstruction with Manhattan assumption

Image-based 3D scene reconstruction and exploration in augmented reality

Data-Driven 3D Reconstruction of Dressed Humans From Sparse Views

Quasi-Globally Optimal and Efficient Vanishing Point Estimation in Manhattan World.

Planar Reconstruction of Indoor Scenes from Sparse Views and Relative Camera Poses

PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo