Abstract:In this paper, we present a new method to modify the appearance of a face image by manipulating the illumination condition, when the face geometry and albedo information is unknown. This problem is particularly difficult when there is only a single image of the subject available. Recent research demonstrates that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace using a spherical harmonic representation. Moreover, morphable models are statistical ensembles of facial properties such as shape and texture. In this paper, we integrate spherical harmonics into the morphable model framework by proposing a 3D spherical harmonic basis morphable model (SHBMM). The proposed method can represent a face under arbitrary unknown lighting and pose simply by three low-dimensional vectors, i.e., shape parameters, spherical harmonic basis parameters, and illumination coefficients, which are called the SHBMM parameters. However, when the image was taken under an extreme lighting condition, the approximation error can be large, thus making it difficult to recover albedo information. In order to address this problem, we propose a subregion-based framework that uses a Markov random field to model the statistical distribution and spatial coherence of face texture, which makes our approach not only robust to extreme lighting conditions, but also insensitive to partial occlusions. The performance of our framework is demonstrated through various experimental results, including the improved rates for face recognition under extreme lighting conditions.

A probabilistic approach to realistic face synthesis with a single uncalibrated image

A synthesis method for personalized 3D face reconstruction

Realistic Face Reenactment Via Self-Supervised Disentangling of Identity and Pose

Realistic Reconstruction of Human Face Based on Images

Automatic 3D Face Modeling Using 2D Active Appearance Models

Realistic 3D Face Modeling by Fusing Multiple 2D Images

Realistic and Efficient Face Swapping: A Unified Approach with Diffusion Models

Face Relighting from a Single Image under Arbitrary Unknown Lighting Conditions

Practical Face Reconstruction via Differentiable Ray Tracing

3DFaceNet: Real-time Dense Face Reconstruction via Synthesizing Photo-realistic Face Images

Synthesis of Realistic Human Faces from CT Image and Color Photos

Cafca: High-quality Novel View Synthesis of Expressive Faces from Casual Few-shot Captures

Towards Realistic Generative 3D Face Models

Photo-realistic Face Images Synthesis for Learning-based Fine-scale 3D Face Reconstruction.

CNN-Based Real-Time Dense Face Reconstruction with Inverse-Rendered Photo-Realistic Face Images.

Digi2Real: Bridging the Realism Gap in Synthetic Data Face Recognition via Foundation Models

Fake It Without Making It: Conditioned Face Generation for Accurate 3D Face Reconstruction

Realistic talking face synthesis with geometry-aware feature transformation

Efficient 3D Reconstruction for Face Recognition

Realistic 3-D human face synthesis based on a deformable face model

Analysis-by-synthesis Approach for Facial Modeling Based on Orthogonal Images