Abstract:Visible facial images provide geometric and appearance patterns of facial expressions and are sensitive to illumination changes. Thermal facial images record facial temperature distribution and are robust to light conditions. Therefore, expression recognition is enhanced by visible and thermal image fusion. In most cases, only visible images are available due to the widespread popularity of visible cameras and the high cost of thermal cameras. Thus, we propose a novel visible expression recognition method by using thermal infrared (IR) data as privileged information, which is only available during training. Specifically, we first learn a deep model for visible images and thermal images. Then we use the learned feature representations to train support vector machine (SVM) classifiers for expression classification. We jointly refine the deep models as well as the SVM classifiers for both thermal images and visible images by imposing the constraint that the outputs of the SVM classifiers from two views are similar. Thermal IR images during training are then exploited to construct better facial representations and expression classifiers from visible images. We extend the proposed thermal augmented expression recognition method for partially unpaired data, acknowledging that visible images and thermal images maybe not be recorded synchronously. Experimental resulton the MAHNOB laughter database demonstrate that the proposed thermal augmented expression recognition method can effectively exploit thermal IR images’ supplementary role for visible facial expression recognition during training to obtain better facial representations and a better visible expression classifier. The proposed thermal augmented expression recognition method achieves state-of-the-art expression recognition performance for both paired and unpaired facial images.

Face Recognition Algorithm by Fusing Thermal Infrared and Visible Images

A Fusion Framework For Face Recognition Under Varying Illumination Based On Multi-Scale Analysis

Image fusion for face recognition

Decision-level Fusion of Infrared and Visible Images for Face Recognition

Fusion of Visible and Infrared Imagery for Face Recognition

Expression Recognition from Visible Images with the Help of Thermal Images

Multi-feature Fusion for Thermal Face Recognition

Performance Evaluation of Infrared and Visible Image Fusion Algorithms for Face Recognition

Fusion of visible and thermal images for facial expression recognition

Fuzzy Fusion for Face Recognition

Multisource Image Fusion for Face Recognition Based on Decision Level

Fusion of Wavelet Coefficients from Visual and Thermal Face Images for Human Face Recognition - A Comparative Study

Enhancing thermal facial recognition leveraging large datasets and hybrid algorithms

Good match exploration for thermal infrared face recognition based on YWF-SIFT with multi-scale fusion

Nonlinear fusion for face recognition using fuzzy integral

Fusion of Infrared and Visible Image Based on HIS and Wavelet Transform

Face recognition based on support vector machine fusion and wavelet transform

Infrared and Visible Image Fusion Using Multi-Scale Edge-Preserving Decomposition and Multiple Saliency Features

Thermal Augmented Expression Recognition

Visible-Information-Aided Eyeglasses Removing for Thermal Image Reconstruction

Infrared and Visible Image Fusion with Hybrid Image Filtering