Abstract:Photometric stereo recovers three-dimensional (3D) object surface normal from multiple images under different illumination directions. Traditional photometric stereo methods suffer from the problem of non-Lambertian surfaces with general reflectance. By leveraging deep neural networks, learning-based methods are capable of improving the surface normal estimation under general non-Lambertian surfaces. These state-of-the-art learning-based methods however do not associate surface normal with reconstructed images and, therefore, they cannot explore the beneficial effect of such association on the estimation of the surface normal. In this paper, we specifically exploit the positive impact of this association and propose a novel dual regression network for both fine surface normals and arbitrary reconstructed images in calibrated photometric stereo. Our work unifies the 3D reconstruction and rendering tasks in a deep learning framework, with the explorations including: 1. generating specified reconstructed images under arbitrary illumination directions, which provides more intuitive perception of the reflectance and is extremely useful for visual applications, such as virtual reality, and 2. our dual regression scheme introduces an additional constraint on observed images and reconstructed images, which forms a closed-loop to provide additional supervision. Experiments show that our proposed method achieves accurate reconstructed images under arbitrarily specified illumination directions and it significantly outperforms the state-of-the-art learning-based single regression methods in calibrated photometric stereo.

Normal and Visibility Estimation of Human Face from a Single Image

Physics-Based Appearance and Illumination Estimation from a Single Face Image

Normalization of face illumination using basic knowledge and information extracted from a single image

Illumination Normalization of Face Image Based on Illuminant Direction Estimation and Improved Retinex

Face Recognition Based on Illumination Normalization in Frequency-Domain

Normalization of Face Illumination Based on Large-and Small-Scale Features

Shadow Compensation and Illumination Normalization of Face Image

SPLiT: Single Portrait Lighting Estimation via a Tetrad of Face Intrinsics

Improving 3D Face Details Based on Normal Map of Hetero-source Images

Face Recognition Under Generic Illumination Based On Harmonic Relighting

Deep Surface Normal Estimation with Hierarchical RGB-D Fusion

Illumination Normalization Using Self-lighting Ratios for 3D2D Face Recognition

Surface Normals in the Wild

Recovering Surface Normal and Arbitrary Images: A Dual Regression Network for Photometric Stereo.

Faces as Lighting Probes via Unsupervised Deep Highlight Extraction

Learning Visibility Field for Detailed 3D Human Reconstruction and Relighting

3D Reconstruction of Human Face from an Input Image under Random Lighting Condition.

Robust Geometry and Reflectance Disentanglement for 3D Face Reconstruction from Sparse-view Images

Face Illumination Normalization Based on Generative Adversarial Network

Facial Depth and Normal Estimation using Single Dual-Pixel Camera

Spherical Harmonic Based Linear Face De-Lighting and Compensation