Abstract:The task of near-infrared and visual (NIR-VIS) face recognition refers to matching face data from different modalities, which has broad application prospects in areas such as multimedia information retrieval and criminal investigation. However, it remains a challenging task due to high intra-class variations and small-scale NIR-VIS dataset. In this paper, we propose a novel approach called Dual Adversarial Disentanglement and deep Representation Decorrelation (DADRD) to solve the NIR-VIS matching problem. In order to reduce the gap between NIR-VIS images, three key components are designed for DADRD model, including Cross-modal Margin (CmM) loss, Dual Adversarial Disentangled Variations (DADV) and Deep Representation Decorrelation (DRD). Firstly, the CmM loss captures within- and between-class information of the data, and it further reduces modality difference by a center-variation item. Secondly, the Mixed Facial Representation (MFR) layer of the backbone network is divided into three parts: the identity-related layer, the modality-related layer and the residual-related layer. The DADV is designed to reduce the intra-class variations, which consists of Adversarial Disentangled Modality Variations (ADMV) and Adversarial Disentangled Residual Variations (ADRV). Specifically, the ADMV and ADRV aim at eliminating spectrum variations and residual variations (i.e., lighting, pose, expression, occlusion, etc) respectively via an adversarial mechanism. Finally, we impose a DRD on the three decomposed features to make them irrelevant to each other, which can more effectively separate the three component information and enhance feature representations. In particular, we develop a Joint Three-stage Optimization (JTsO) strategy to effectively optimize the network. The joint formulation leads to the purification of identity information and the disentanglement of within-class variation information. Extensive experiments have been carried out on three challenging datasets, and the results demonstrate the effectiveness of our method.

A Bemd Based Muti-Layer Face Matching: From Near Infrared To Visual Images

Adaptive Illumination-Invariant Face Recognition Via Local Nonlinear Multi-Layer Contrast Feature.

Face recognition method under varying illumination based on FABEMD

Design and implementation of a bimodal face recognition system

Face Recognition Based on Local Binary Pattern and Improved Pairwise-constrained Multiple Metric Learning

A Robust Method For Near Infrared Face Recognition Based On Extended Local Binary Pattern

A BEMD Based Normalization Method for Face Recognition under Variable Illuminations

Near infrared face recognition using Zernike moments and Hermite kernels

Illumination Invariant Face Recognition Using Fabemd Decomposition with Detail Measure Weight.

Matching NIR face to VIS face using multi-feature based MSDA

3-D Face Recognition Using Elbp-Based Facial Description and Local Feature Hybrid Matching

Multisource Image Fusion for Face Recognition Based on Decision Level

Face Recognition Using Early Biologically Inspired Features

Dual Adversarial Disentanglement and Deep Representation Decorrelation for NIR-VIS Face Recognition

A NIR-to-VIS face recognition via part adaptive and relation attention module

A Multi-modal 2D + 3D Face Recognition Method with a Novel Local Feature Descriptor

Extended Local Binary Patterns for Face Recognition

Matching NIR Face to VIS Face Using Transduction

Domain Discrepancy Elimination and Mean Face Representation Learning for NIR-VIS Face Recognition

Illumination Invariant Face Recognition Using Near-Infrared Images

Tensor Bidirectional Reflectance Distribution Function Model Graph Based Multi-Spectral Face Recognition