Abstract:Photometric stereo is a powerful method for obtaining per-pixel surface normals from differently illuminated images of an object. While several methods address photometric stereo with different image (or light) counts ranging from one to two to a hundred, very few focus on learning optimal lighting configuration. Finding an optimal configuration is challenging due to the vast number of possible lighting directions. Moreover, exhaustively sampling all possibilities is impractical due to time and resource constraints. Photometric stereo methods have demonstrated promising performance on existing datasets, which feature limited light directions sparsely sampled from the light space. Therefore, can we optimally utilize these datasets for illumination planning? In this work, we introduce LIPIDS - Learning-based Illumination Planning In Discretized light Space to achieve minimal and optimal lighting configurations for photometric stereo under arbitrary light distribution. We propose a Light Sampling Network (LSNet) that optimizes lighting direction for a fixed number of lights by minimizing the normal loss through a normal regression network. The learned light configurations can directly estimate surface normals during inference, even using an off-the-shelf photometric stereo method. Extensive qualitative and quantitative analyses on synthetic and real-world datasets show that photometric stereo under learned lighting configurations through LIPIDS either surpasses or is nearly comparable to existing illumination planning methods across different photometric stereo backbones.

Learning to Minify Photometric Stereo

Summary Study of Data-Driven Photometric Stereo Methods

Learning Inter- and Intra-frame Representations for Non-Lambertian Photometric Stereo

Robust photometric stereo using structural light sources

Minifying photometric stereo via knowledge distillation-based feature translation.

Efficient Photometric Stereo Using Kernel Regression

LIPIDS: Learning-based Illumination Planning In Discretized (Light) Space for Photometric Stereo

Photometric Stereo Based on Multiple Kernel Learning

Learning Conditional Photometric Stereo with High-Resolution Features

Deep Learning Methods for Calibrated Photometric Stereo and Beyond

Photometric Stereo Using Internet Images.

Deep Photometric Stereo for Non-Lambertian Surfaces

DeepPS2: Revisiting Photometric Stereo Using Two Differently Illuminated Images

Deep Photometric Stereo Network

Learning Photometric Stereo Via Manifold-based Mapping.

GR-PSN: Learning to Estimate Surface Normal and Reconstruct Photometric Stereo Images.

Dense 3D Reconstruction Based on Photometric Stereo with Unknown Light Source via Energy Minimization Framework

A Coarse-to-Fine Reconstruction Framework for Non-Lambertian Photometric Stereo

Learning Deep Photometric Stereo Network with Reflectance Priors

Deep Uncalibrated Photometric Stereo via Inter-Intra Image Feature Fusion

Surface normal estimation from optimized and distributed light sources using DNN-based photometric stereo