Abstract:Near-Infrared and Visible light (NIR-VIS) face recognition methods have achieved remarkable success in the fields of security surveillance, criminal investigation, and multimedia information retrieval. But the existing methods heavily rely on carefully annotated labels, leading to expensive manual labelling consumption and deployment flexibility. This motivates us to design unsupervised methods to address NIR-VIS recognition without relying on label information. To this end, we propose a novel homogeneous-to-HEterogeneous learning and Residual-invariant Enhancement (HERE) network for Unsupervised NIR-VIS Heterogeneous Face Recognition (NIR-VIS-UHFR). As the name suggests, the optimization of HERE follow a ”homogeneous-to-heterogeneous learning” strategy to fully explore complementary and common semantic information across different modalities. During the homogeneous learning phase, Modality-Adversarial Contrastive Learning (MACL) leverages the collaboration of modality contrastive learning and adversarial learning. On the one hand, MACL learns compact and discriminative intra-modal representations for NIR and VIS data, respectively. On the other hand, MACL guarantees that NIR-VIS data conform to the common feature distribution in a shared feature space, effectively reducing modal differences even in the absence of identity information between modalities. In the heterogeneous learning phase, K-reciprocal-Encoding-based Cross-modal Labeling (KECL) is introduced as robust pseudo label estimation to fully explore cross-modal relationships and group cross-modal features into clusters. With the pseudo labels provided by KECL, Refined cross-modal Contrastive Learning (RCL) is developed with modality-invariant averaging initialization and dynamic focus weighting strategies to extract modality-invariant features. Finally, Residual-invariant Representations Enhancement (RRE) mines partial features under the cross-modal face for robust matching. Compared to supervised methods, our unsupervised HERE demonstrates comparable performance on multiple datasets, greater scalability and practicality in deployment by reducing data acquisition requirements and costs.

Near-infrared and visible light face recognition: a comprehensive survey

Near infrared face recognition using Zernike moments and Hermite kernels

Towards Nir-Vis Masked Face Recognition

Illumination Invariant Face Recognition Using Near-Infrared Images

Transductive VIS-NIR face matching

A NIR-to-VIS face recognition via part adaptive and relation attention module

Infrared face recognition: a comprehensive review of methodologies and databases

Matching NIR Face to VIS Face Using Transduction

IR and visible light face recognition

Learning mappings for face synthesis from near infrared to visual light images

Cross-Modal and Multi-Attribute Face Recognition: A Benchmark

Design and implementation of a bimodal face recognition system

SWU-NIRPV: a Near-Infrared Pose Variation Face Database and Pose-Invariant Face Recognition

Fusion of Visible and Infrared Imagery for Face Recognition

Dual Adversarial Disentanglement and Deep Representation Decorrelation for NIR-VIS Face Recognition

Cross-spectral Face Completion for NIR-VIS Heterogeneous Face Recognition

Adversarial Cross-Spectral Face Completion for NIR-VIS Face Recognition

Domain Discrepancy Elimination and Mean Face Representation Learning for NIR-VIS Face Recognition

Not Afraid of the Dark: NIR-VIS Face Recognition via Cross-spectral Hallucination and Low-rank Embedding

Learning Mappings for synthesis from near infrared to light images

Unsupervised NIR-VIS Face Recognition Via Homogeneous-to-Heterogeneous Learning and Residual-Invariant Enhancement