Abstract:Over the course of the last decade, infrared (IR) and particularly thermal IR imaging based face recognition has emerged as a promising complement to conventional, visible spectrum based approaches which continue to struggle when applied in practice. While inherently insensitive to visible spectrum illumination changes, IR data introduces specific challenges of its own, most notably sensitivity to factors which affect facial heat emission patterns, e.g. emotional state, ambient temperature, and alcohol intake. In addition, facial expression and pose changes are more difficult to correct in IR images because they are less rich in high frequency detail which is an important cue for fitting any deformable model. In this paper we describe a novel method which addresses these major challenges. Specifically, when comparing two thermal IR images of faces, we mutually normalize their poses and facial expressions by using an active appearance model (AAM) to generate synthetic images of the two faces with a neutral facial expression and in the same view (the average of the two input views). This is achieved by piecewise affine warping which follows AAM fitting. A major contribution of our work is the use of an AAM ensemble in which each AAM is specialized to a particular range of poses and a particular region of the thermal IR face space. Combined with the contributions from our previous work which addressed the problem of reliable AAM fitting in the thermal IR spectrum, and the development of a person-specific representation robust to transient changes in the pattern of facial temperature emissions, the proposed ensemble framework accurately matches faces across the full range of yaw from frontal to profile, even in the presence of scale variation (e.g. due to the varying distance of a subject from the camera). The effectiveness of the proposed approach is demonstrated on the largest public database of thermal IR images of faces and a newly acquired data set of thermal IR motion videos. Our approach achieved perfect recognition performance on both data sets, significantly outperforming the current state of the art methods even when they are trained with multiple images spanning a range of head views.

Deep Cross Polarimetric Thermal-to-visible Face Recognition

The Method Of Infrared Polariametric Imaging

Polarimetric Thermal to Visible Face Verification via Attribute Preserved Synthesis

Polarimetric Thermal to Visible Face Verification via Self-Attention Guided Synthesis

Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks

Generative Adversarial Network-based Synthesis of Visible Faces from Polarimetric Thermal Faces

Polar Fusion Technique Analysis for Evaluating the Performances of Image Fusion of Thermal and Visual Images for Human Face Recognition

Deep features class activation map for thermal face detection and tracking

Enhancing thermal facial recognition leveraging large datasets and hybrid algorithms

Learning Domain and Pose Invariance for Thermal-to-Visible Face Recognition

Cross-Domain Identification for Thermal-to-Visible Face Recognition

A unified framework for thermal face recognition

Illumination-invariant face recognition from a single image across extreme pose using a dual dimension AAM ensemble in the thermal infrared spectrum

Thermal to Visible Face Recognition Using Deep Autoencoders

Influence of Thermal Imagery Resolution on Accuracy of Deep Learning based Face Recognition

A Synthesis-Based Approach for Thermal-to-Visible Face Verification

Deep Cross-View Reconstruction GAN Based on Correlated Subspace for Multi-View Transformation

Face emotion recognition based on infrared thermal imagery by applying machine learning and parallelism

Unpaired Multimodal Facial Expression Recognition.

Deep Heterogeneous Feature Fusion for Template-Based Face Recognition

HyperFace: A Deep Fusion Model for Hyperspectral Face Recognition