Abstract:We address the problem of robust normal reconstruction by dense photometric stereo, in the presence of complex geometry, shadows, highlight, transparencies, variable attenuation in light intensities, and inaccurate estimation in light directions. The input is a dense set of noisy photometric images, conveniently captured by using a very simple set-up consisting of a digital video camera, a reflective mirror sphere, and a handheld spotlight. We formulate the dense photometric stereo problem as a Markov network and investigate two important inference algorithms for Markov Random Fields (MRFs)--graph cuts and belief propagation--to optimize for the most likely setting for each node in the network. In the graph cut algorithm, the MRF formulation is translated into one of energy minimization. A discontinuity-preserving metric is introduced as the compatibility function, which allows alpha-expansion to efficiently perform the maximum a posteriori (MAP) estimation. Using the identical dense input and the same MRF formulation, our tensor belief propagation algorithm recovers faithful normal directions, preserves underlying discontinuities, improves the normal estimation from one of discrete to continuous, and drastically reduces the storage requirement and running time. Both algorithms produce comparable and very faithful normals for complex scenes. Although the discontinuity-preserving metric in graph cuts permits efficient inference of optimal discrete labels with a theoretical guarantee, our estimation algorithm using tensor belief propagation converges to comparable results, but runs faster because very compact messages are passed and combined. We present very encouraging results on normal reconstruction. A simple algorithm is proposed to reconstruct a surface from a normal map recovered by our method. With the reconstructed surface, an inverse process, known as relighting in computer graphics, is proposed to synthesize novel images of the given scene under user-specified light source and direction. The synthesis is made to run in real time by exploiting the state-of-the-art graphics processing unit (GPU). Our method offers many unique advantages over previous relighting methods and can handle a wide range of novel light sources and directions.

Photometric Stereo Via Random Sampling and Tensor Robust Principal Component Analysis

Non-Lambertian photometric stereo by color and noise analysis

Robust photometric stereo using structural light sources

Summary Study of Data-Driven Photometric Stereo Methods

On accurate recovery of 3D surface normal using minimum 2D images

A Non-Lambertian Photometric Stereo under Perspective Projection

Robust Photometric Stereo Via Low-Rank Matrix Completion And Recovery

A Simple 3D Reconstruction Method Based on Single Camera

Learning Inter- and Intra-frame Representations for Non-Lambertian Photometric Stereo

Dense 3D Reconstruction Based on Photometric Stereo with Unknown Light Source via Energy Minimization Framework

Three-Dimensional Reconstruction from Single Image Base on Combination of CNN and Multi-Spectral Photometric Stereo

Efficient Photometric Stereo Using Kernel Regression

Robust 3D Reconstruction with an RGB-D Camera

Dense Photometric Stereo: A Markov Random Field Approach

Estimating High-resolution Surface Normals via Low-resolution Photometric Stereo Images

Patch-Based Uncalibrated Photometric Stereo under Natural Illumination.

High-Quality RGB-D Reconstruction via Multi-View Uncalibrated Photometric Stereo and Gradient-SDF

The 3D Reconstruction of Face Model with Active Structured Light and Stereo Vision Fusion

A Probabilistic Approach for Stereo 3D Point Cloud Reconstruction from Airborne Single-Channel Multi-Aspect SAR Image Sequences

Convex Optimization Based Low-Rank Matrix Completion and Recovery for Photometric Stereo and Factor Classification

3D Face Reconstruction Using Color Photometric Stereo with Uncalibrated Near Point Lights