Abstract:Cross-spectral face hallucination is an intuitive way to mitigate the modality discrepancy in Heterogeneous Face Recognition (HFR). However, due to imaging differences, the hallucination inevitably suffers from a shape misalignment between paired heterogeneous images. Rather than building complicated architectures to circumvent the problem like previous works, we propose a simple yet effective method called Shape Alignment FacE (SAFE). Specifically, given an image, we align its shape to that of the paired one under the assistance of a 3D face model. The produced aligned pair enables us to train a lightweight generator that solely concentrates on spectrum translation with a pixel-wise supervision. However, since the 3D face model is powerless to attributes like the hair and glasses, there are still pixel discrepancies between the aligned pair. Given that, in the image space, we introduce a probabilistic pixel-wise loss that incorporates the discrepancies into a probabilistic distribution. Moreover, in order to alleviate the influence of the shape misalignment on spectrum translation, a spectrum optimal transport is performed in a shape-irrelevant latent space. Note that, in the final inference phase, except the lightweight generator, all other auxiliary modules are discarded. In addition to superior performance in qualitative synthesis and quantitative recognition, extensive experiments on 6 datasets demonstrate that our method also gains other two distinct advantages over existing state-of-the-art counterparts. The first is using a more lightweight generator. Compared with the state-of-the-art method, our method can achieve higher recognition results with 128x fewer parameters and 63x fewer FLOPswith only 4.58 ms latency on a single TITAN-XP. The second is training on low-shot datasets such as Oulu-CASIA NIR-VIS that just contains 1,920 images from 20 identities. To the best of our knowledge, we are the first that can performwell on such a small-scale dataset. These advantages make our method more practical in the real world and further push boundaries of heterogeneous face recognition.

Contrastive Learning with Hallucinating Data for Long-Tailed Face Recognition.

Adaptive Illumination-Invariant Face Recognition Via Local Nonlinear Multi-Layer Contrast Feature.

Bt-Vmf Contrastive and Collaborative Learning for Long-Tailed Visual Recognition

Targeted Supervised Contrastive Learning for Long-Tailed Recognition

Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition

Long-Tailed Recognition by Mutual Information Maximization between Latent Features and Ground-Truth Labels

Decoupled Contrastive Learning for Long-Tailed Recognition

SMCL: Saliency Masked Contrastive Learning for Long-tailed Recognition

Contrastive Learning Based Hybrid Networks for Long-Tailed Image Classification

C-Face: Using Compare Face on Face Hallucination for Low-Resolution Face Recognition

Unequal-Training for Deep Face Recognition with Long-Tailed Noisy Data

Balanced Contrastive Learning for Long-Tailed Visual Recognition

Face Hallucination Via Multiple Feature Learning with Hierarchical Structure

Probabilistic Contrastive Learning for Long-Tailed Visual Recognition

Deep Balanced Learning for Long-tailed Facial Expressions Recognition

Learning Fractional Orthogonal Latent Consistent Features for Face Hallucination and Recognition

Towards Lightweight Pixel-Wise Hallucination for Heterogeneous Face Recognition

Adversarial Discriminative Heterogeneous Face Recognition.

Range Loss for Deep Face Recognition with Long-tail

Improvised contrastive loss for improved face recognition in open-set nature

Asynchronous Modeling: A Dual-phase Perspective for Long-Tailed Recognition