Abstract:Reconstructing the 3D shape of an object using several images under different light sources is a very challenging task, especially when realistic assumptions such as light propagation and attenuation, perspective viewing geometry and specular light reflection are considered. Many of works tackling Photometric Stereo (PS) problems often relax most of the aforementioned assumptions. Especially they ignore specular reflection and global illumination effects. In this work, we propose a CNN-based approach capable of handling these realistic assumptions by leveraging recent improvements of deep neural networks for far-field Photometric Stereo and adapt them to the point light setup. We achieve this by employing an iterative procedure of point-light PS for shape estimation which has two main steps. Firstly we train a per-pixel CNN to predict surface normals from reflectance samples. Secondly, we compute the depth by integrating the normal field in order to iteratively estimate light directions and attenuation which is used to compensate the input images to compute reflectance samples for the next iteration. Our approach sigificantly outperforms the state-of-the-art on the DiLiGenT real world dataset. Furthermore, in order to measure the performance of our approach for near-field point-light source PS data, we introduce LUCES the first real-world 'dataset for near-fieLd point light soUrCe photomEtric Stereo' of 14 objects of different materials were the effects of point light sources and perspective viewing are a lot more significant. Our approach also outperforms the competition on this dataset as well. Data and test code are available at the project page.

DiLiGenT-Pi: Photometric Stereo for Planar Surfaces with Rich Details - Benchmark Dataset and Beyond

DiLiGenT102: A Photometric Stereo Benchmark Dataset with Controlled Shape and Material Variation

A Benchmark Dataset and Evaluation for Non-Lambertian and Uncalibrated Photometric Stereo

Summary Study of Data-Driven Photometric Stereo Methods

DiLiGenRT: A Photometric Stereo Dataset with Quantified Roughness and Translucency

A Non-Lambertian Photometric Stereo under Perspective Projection

LUCES-MV: A Multi-View Dataset for Near-Field Point Light Source Photometric Stereo

A Novel Learning Based Non-Lambertian Photometric Stereo Method for Pixel-Level Normal Reconstruction of Polished Surfaces

Multi-view photometric stereo of non-Lambertian surface under general illuminations

IRS: A Large Naturalistic Indoor Robotics Stereo Dataset to Train Deep Models for Disparity and Surface Normal Estimation

LUCES: A Dataset for Near-Field Point Light Source Photometric Stereo

A Method for 3D Reconstruction Based on Photometric Stereo Image

Multi-View Photometric Stereo: A Robust Solution and Benchmark Dataset for Spatially Varying Isotropic Materials

A CNN Based Approach for the Point-Light Photometric Stereo Problem

Learning Inter- and Intra-frame Representations for Non-Lambertian Photometric Stereo

Scalable, Detailed and Mask-Free Universal Photometric Stereo

A Coarse-to-Fine Reconstruction Framework for Non-Lambertian Photometric Stereo

An evaluation of Deep Learning based stereo dense matching dataset shift from aerial images and a large scale stereo dataset

Booster: a Benchmark for Depth from Images of Specular and Transparent Surfaces

Photometric Stereo for General Reflectance and Lighting