Abstract:In this paper, we present a new method to modify the appearance of a face image by manipulating the illumination condition, when the face geometry and albedo information is unknown. This problem is particularly difficult when there is only a single image of the subject available. Recent research demonstrates that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace using a spherical harmonic representation. Moreover, morphable models are statistical ensembles of facial properties such as shape and texture. In this paper, we integrate spherical harmonics into the morphable model framework by proposing a 3D spherical harmonic basis morphable model (SHBMM). The proposed method can represent a face under arbitrary unknown lighting and pose simply by three low-dimensional vectors, i.e., shape parameters, spherical harmonic basis parameters, and illumination coefficients, which are called the SHBMM parameters. However, when the image was taken under an extreme lighting condition, the approximation error can be large, thus making it difficult to recover albedo information. In order to address this problem, we propose a subregion-based framework that uses a Markov random field to model the statistical distribution and spatial coherence of face texture, which makes our approach not only robust to extreme lighting conditions, but also insensitive to partial occlusions. The performance of our framework is demonstrated through various experimental results, including the improved rates for face recognition under extreme lighting conditions.

Virtual Face Image Generation for Illumination and Pose Insensitive Face Recognition

Pose Invariant Face Recognition under Arbitrary Illumination Based on 3D Face Reconstruction

Face Detection under Variable Lighting Based on Resample by Face Relighting

Image Synthesis and Face Recognition Based on 3D Face Model and Illumination Model.

Generating Face Images under Multiple Illuminations Based on a Single Front-Lighted Sample Without 3D Models.

A survey of virtual sample generation technology for face recognition

A Face Wrapping Method Based on Pose-Specific Shape Eigenspace

Recognition of head-and-shoulder face image using virtual frontal-view image

Illumination compensation and enhancement for face recognition

Face Re-Lighting From A Single Image Under Harsh Lighting Conditions

Face Relighting from a Single Image under Arbitrary Unknown Lighting Conditions

Face Recognition Under Generic Illumination Based On Harmonic Relighting

Scalable Illumination Robust Face Identification Using Harmonic Representation

Illumination Normalization For Robust Face Recognition Against Varying Lighting Conditions

View synthesis from Infrared-visual fused 3D model for face recognition

VIGFace: Virtual Identity Generation for Privacy-Free Face Recognition

A novel gradient synthesis-based illumination formalization method for face recognition under varying lighting conditions

Towards High Fidelity Face Relighting with Realistic Shadows

Illumination Ratio Image: Synthesizing And Recognition With Varying Illuminations

Face Image Illumination Processing Based on GAN with Dual Triplet Loss.

Face Samples Re-Lighting for Detection Based on the Harmonic Images