Abstract:We address the problem of robust normal reconstruction by dense photometric stereo, in the presence of complex geometry, shadows, highlight, transparencies, variable attenuation in light intensities, and inaccurate estimation in light directions. The input is a dense set of noisy photometric images, conveniently captured by using a very simple set-up consisting of a digital video camera, a reflective mirror sphere, and a handheld spotlight. We formulate the dense photometric stereo problem as a Markov network and investigate two important inference algorithms for Markov Random Fields (MRFs)--graph cuts and belief propagation--to optimize for the most likely setting for each node in the network. In the graph cut algorithm, the MRF formulation is translated into one of energy minimization. A discontinuity-preserving metric is introduced as the compatibility function, which allows alpha-expansion to efficiently perform the maximum a posteriori (MAP) estimation. Using the identical dense input and the same MRF formulation, our tensor belief propagation algorithm recovers faithful normal directions, preserves underlying discontinuities, improves the normal estimation from one of discrete to continuous, and drastically reduces the storage requirement and running time. Both algorithms produce comparable and very faithful normals for complex scenes. Although the discontinuity-preserving metric in graph cuts permits efficient inference of optimal discrete labels with a theoretical guarantee, our estimation algorithm using tensor belief propagation converges to comparable results, but runs faster because very compact messages are passed and combined. We present very encouraging results on normal reconstruction. A simple algorithm is proposed to reconstruct a surface from a normal map recovered by our method. With the reconstructed surface, an inverse process, known as relighting in computer graphics, is proposed to synthesize novel images of the given scene under user-specified light source and direction. The synthesis is made to run in real time by exploiting the state-of-the-art graphics processing unit (GPU). Our method offers many unique advantages over previous relighting methods and can handle a wide range of novel light sources and directions.

Prior surface model guided dense reconstruction via graph-cuts

A clustering approach to free form surface reconstruction from multi-view range images

From Stereo Pair to 3D Face Model

CRF-Based Reconstruction from Narrow-Baseline Image Sequences.

High-Quality Depth Recovery Via Interactive Multi-view Stereo

DP-MVS: Detail Preserving Multi-View Surface Reconstruction of Large-Scale Scenes

Progressive surface reconstruction from images using a local prior

ResDepth: A Deep Residual Prior For 3D Reconstruction From High-resolution Satellite Images

NeuSurf: On-Surface Priors for Neural Surface Reconstruction from Sparse Input Views

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

PR-NeuS: A Prior-based Residual Learning Paradigm for Fast Multi-view Neural Surface Reconstruction

Surface Reconstruction By Propagating 3d Stereo Data In Multiple 2d Images

Surface-Centric Modeling for High-Fidelity Generalizable Neural Surface Reconstruction

Accurate 3D reconstruction via surface-consistency

Reconstructing Surfaces for Sparse Point Clouds with On-Surface Priors

A Stochastic Algorithm for 3D Scene Segmentation and Reconstruction

Dense Photometric Stereo: A Markov Random Field Approach

HQP-MVS:High-Quality Plane Priors Assisted Multi-View Stereo for Low-Textured Areas

High Fidelity Aggregated Planar Prior Assisted PatchMatch Multi-View Stereo

Rate-constrained 3D surface estimation from noise-corrupted multiview depth videos

Real-time Dense Reconstruction of Tissue Surface from Stereo Optical Video