Abstract:Photometric stereo (PS) endeavors to ascertain surface normals using shading clues from photometric images under various illuminations. Recent deep learning-based PS methods often overlook the complexity of object surfaces. These neural network models, which exclusively rely on photometric images for training, often produce blurred results in high-frequency regions characterized by local discontinuities, such as wrinkles and edges with significant gradient changes. To address this, we propose the Image Gradient-Aided Photometric Stereo Network (IGA-PSN), a dual-branch framework extracting features from both photometric images and their gradients. Furthermore, we incorporate an hourglass regression network along with supervision to regularize normal regression. Experiments on DiLiGenT benchmarks show that IGA-PSN outperforms previous methods in surface normal estimation, achieving a mean angular error of 6.46 while preserving textures and geometric shapes in complex regions.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the problems encountered by existing deep - learning - based Photometric Stereo (PS) methods when dealing with complex surface structures. Specifically, these problems include: 1. **Blurring in high - frequency regions**: Existing PS methods are prone to produce blurry results when dealing with high - frequency regions with complex structures (such as wrinkles and edges). These methods usually rely only on photometric images for training and it is difficult to capture local discontinuity features. 2. **Insufficient discrimination of frequency - band information**: Existing PS networks do not distinguish high - and low - frequency information well when extracting features, resulting in insufficient attention to high - frequency information and affecting the overall performance. 3. **Limitations of the loss function**: Traditional PS methods usually use cosine similarity as the loss function, which can only consider the average angular difference and ignores regions with rich surface details, resulting in overly smooth and blurry outputs. To solve the above problems, the authors propose a new framework named **Image Gradient - Aided Photometric Stereo Network (IGA - PSN)**. The main improvements of this framework include: - **Introduction of image - gradient assistance**: By introducing image - gradient information, the ability to capture high - frequency regions (such as wrinkles and edges) is enhanced. - **Dual - branch network structure**: A multi - path dual - branch parallel network is designed to process high - and low - frequency regions respectively, and the gradient information of the input image is deeply mined through a structural - gradient extractor, thereby achieving high - quality surface normal recovery. - **Attention - feature - fusion module**: An attention - feature - fusion module is developed to enable the network to focus on specific local regions and adaptively aggregate model - specific features to optimize feature extraction. - **Improved loss function**: In addition to the traditional cosine - similarity loss, a gradient - error loss is also introduced to ensure that the network can better focus on structural differences and ensure clear and accurate surface - normal recovery. - **Hourglass regression module**: An Hourglass regression module with multi - level supervision is designed to participate in normal regression in an iterative manner to maximize the utilization of global information. Experimental results show that IGA - PSN performs excellently in the DiLiGenT benchmark test, achieving an average angular error of 6.46 degrees and preserving textures and geometric shapes in complex regions.

Image Gradient-Aided Photometric Stereo Network

Summary Study of Data-Driven Photometric Stereo Methods

GR-PSN: Learning to Estimate Surface Normal and Reconstruct Photometric Stereo Images.

NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo

Non-Lambertian photometric stereo by color and noise analysis

GPS-Net: Graph-based Photometric Stereo Network.

Robust photometric stereo using structural light sources

Learning Inter- and Intra-frame Representations for Non-Lambertian Photometric Stereo

Learning Conditional Photometric Stereo with High-Resolution Features

Deep Photometric Stereo Network

PS-Transformer: Learning Sparse Photometric Stereo Network using Self-Attention Mechanism

DPS-Net: Deep Polarimetric Stereo Depth Estimation

NormAttention-PSN: A High-frequency Region Enhanced Photometric Stereo Network with Normalized Attention

Estimating High-resolution Surface Normals via Low-resolution Photometric Stereo Images

Remote Sensing Single-Image Resolution Improvement Using A Deep Gradient-Aware Network with Image-Specific Enhancement

Three-Dimensional Reconstruction from Single Image Base on Combination of CNN and Multi-Spectral Photometric Stereo

Surface normal estimation from optimized and distributed light sources using DNN-based photometric stereo

SPLINE-Net: Sparse Photometric Stereo through Lighting Interpolation and Normal Estimation Networks

Minifying photometric stereo via knowledge distillation-based feature translation.

Efficient Photometric Stereo Using Kernel Regression

Self-Calibrating Deep Photometric Stereo Networks