Abstract:Acquiring high-resolution 3D surface structures is a crucial task in computer vision as it provides more detailed surface textures and clearer structures. Photometric stereo can measure per-pixel surface normals of a 3D object using various shading cues. However, obtaining high-resolution images in a linear response photometric stereo imaging system can be challenging. Additionally, photometric stereo, as a per-pixel reconstruction method, requires higher-resolution surface normal maps to accurately depict complex surface structures, particularly in regions that demand more attention and precise reconstruction. Therefore, measuring high-resolution surface normals via low-resolution photometric stereo images is of great importance. Motivated by these, we propose a Super-resolution Photometric Stereo Network, namely SR-PSN. In order to address the issues of measuring the high-resolution surface normals from low-resolution photometric images, we mainly (1) apply a dual-position threshold normalization pre-processing scheme to effectively handle the spatially-varying reflectance of non-Lambertian surfaces, (2) adopt a local affinity feature module to learn the rich structural representation by explicitly revealing the neighbor relationships, (3) employ a parallel multi-scale feature extractor, which preserves high-resolution representations and deep feature extraction, and (4) propose a shared-weight regressor to handle the multi-scale features, to prevent the model collapsing into learning non-important features related to a certain fixed scale. Extensive ablation experiments validate the effectiveness of our proposed modules. Furthermore, quantitative experiments conducted on public benchmarks demonstrate that SR-PSN outperforms state-of-the-art calibrated photometric stereo methods. Notably, SR-PSN achieves superior results while utilizing photometric stereo images with only half the resolution of other methods. It effectively restores the structure of complex surfaces, producing a high-resolution normal map.

IRS: A Large Naturalistic Indoor Robotics Stereo Dataset to Train Deep Models for Disparity and Surface Normal Estimation

DiLiGenT102: A Photometric Stereo Benchmark Dataset with Controlled Shape and Material Variation

InStereo2K: a large real dataset for stereo matching in indoor scenes

Learning Inter- and Intra-frame Representations for Non-Lambertian Photometric Stereo

DiLiGenT-Pi: Photometric Stereo for Planar Surfaces with Rich Details - Benchmark Dataset and Beyond

A construction method of a large-scale physical rendering 3D semantic segmentation dataset

Hybrid Uncalibrated Near-light Photometric Stereo in Realistic Environment

DrivingStereo: A Large-Scale Dataset for Stereo Matching in Autonomous Driving Scenarios

InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset

FIReStereo: Forest InfraRed Stereo Dataset for UAS Depth Perception in Visually Degraded Environments

An evaluation of Deep Learning based stereo dense matching dataset shift from aerial images and a large scale stereo dataset

Estimating High-resolution Surface Normals via Low-resolution Photometric Stereo Images

Large-Scale Indoor Visual-Geometric Multimodal Dataset and Benchmark for Novel View Synthesis

Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks

A Novel Recurrent Encoder-Decoder Structure for Large-Scale Multi-View Stereo Reconstruction From an Open Aerial Dataset

DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision

LUCES-MV: A Multi-View Dataset for Near-Field Point Light Source Photometric Stereo

A Benchmark Dataset and Evaluation for Non-Lambertian and Uncalibrated Photometric Stereo

UWStereo: A Large Synthetic Dataset for Underwater Stereo Matching

DSEC: A Stereo Event Camera Dataset for Driving Scenarios