Abstract:Near infrared-visible (NIR-VIS) heterogeneous face recognition refers to the process of matching NIR to VIS face images. Current heterogeneous methods try to extend VIS face recognition methods to the NIR spectrum by synthesizing VIS images from NIR images. However, due to self-occlusion and sensing gap, NIR face images lose some visible lighting contents so that they are always incomplete compared to VIS face images. This paper models high resolution heterogeneous face synthesis as a complementary combination of two components, a texture inpainting component and pose correction component. The inpainting component synthesizes and inpaints VIS image textures from NIR image textures. The correction component maps any pose in NIR images to a frontal pose in VIS images, resulting in paired NIR and VIS textures. A warping procedure is developed to integrate the two components into an end-to-end deep network. A fine-grained discriminator and a wavelet-based discriminator are designed to supervise intra-class variance and visual quality respectively. One UV loss, two adversarial losses and one pixel loss are imposed to ensure synthesis results. We demonstrate that by attaching the correction component, we can simplify heterogeneous face synthesis from one-to-many unpaired image translation to one-to-one paired image translation, and minimize spectral and pose discrepancy during heterogeneous recognition. Extensive experimental results show that our network not only generates high-resolution VIS face images and but also facilitates the accuracy improvement of heterogeneous face recognition.

Face synthesis from near-infrared to visual light spectrum using quotient image and kernel-based multifactor analysis

Face Synthesis from Near-Infrared to Visual Light Via Sparse Representation

Face synthesis from low-resolution near-infrared to high-resolution visual light spectrum based on tensor analysis.

Near infrared face recognition using Zernike moments and Hermite kernels

A Fusion Framework For Face Recognition Under Varying Illumination Based On Multi-Scale Analysis

A super-resolution based method to synthesize visual images from near infrared

Learning mappings for face synthesis from near infrared to visual light images

View synthesis from Infrared-visual fused 3D model for face recognition

Multi-View Face Image Synthesis Using Factorization Model

Extracting intrinsic images from multi-spectral

Kernel-Based Multifactor Analysis for Image Synthesis and Recognition

Automatical face beautification based on near-infrared image

Heterogeneous face biometrics based on Guassian weights and invariant features synthesis

An Analysis-by-synthesis Method Based on Sparse Representation for Heterogeneous Face Biometrics

Recovering Facial Intrinsic Images From A Single Input

Near-infrared and visible light face recognition: a comprehensive survey

Learning Mappings for synthesis from near infrared to light images

Cross-spectral Face Completion for NIR-VIS Heterogeneous Face Recognition

Adversarial Cross-Spectral Face Completion for NIR-VIS Face Recognition

Hallucinating Faces From Thermal Infrared Images

Facial Expression Synthesis and Recognition Using a Kernel-Based Factorization Model