Abstract:Deep learning has achieved great success in face recognition, however deep-learned features still have limited invariance to strong intra-personal variations such as large pose changes. It is observed that some facial attributes (e.g. eyebrow thickness, gender) are robust to such variations. We present the first work to systematically explore how the fusion of face recognition features (FRF) and facial attribute features (FAF) can enhance face recognition performance in various challenging scenarios. Despite the promise of FAF, we find that in practice existing fusion methods fail to leverage FAF to boost face recognition performance in some challenging scenarios. Thus, we develop a powerful tensor-based framework which formulates feature fusion as a tensor optimisation problem. It is nontrivial to directly optimise this tensor due to the large number of parameters to optimise. To solve this problem, we establish a theoretical equivalence between low-rank tensor optimisation and a two-stream gated neural network. This equivalence allows tractable learning using standard neural network optimisation tools, leading to accurate and stable optimisation. Experimental results show the fused feature works better than individual features, thus proving for the first time that facial attributes aid face recognition. We achieve state-of-the-art performance on three popular databases: MultiPIE (cross pose, lighting and expression), CASIA NIR-VIS2.0 (cross-modality environment) and LFW (uncontrolled environment).

Safe Facial Multi-Attribute Feature Fusion Analysis Based on Embedded Multi-Task Learning

Fusion Learning Model for Embedded Face Safe Detection and Facial Gesture Analysis

Fusion Learning Model for Mobile Face Safe Detection and Facial Gesture Analysis

Joint Multi-Feature Fusion and Attribute Relationships for Facial Attribute Prediction.

A New Deep Learning Method for Multi-label Facial Expression Recognition Based on Local Constraint Features

Novel Multi-Feature Fusion Facial Aesthetic Analysis Framework.

Learning and Fusing Multimodal Features from and for Multi-task Facial Computing

Feature Level Information Fusion Based Deep Learning

Face detection method fusing multi-layer features

Multi-feature based robust face detection and coarse alignment method via multiple kernel learning

Grouped Multi-Task CNN for Facial Attribute Recognition

Deep Multi-task Multi-label CNN for Effective Facial Attribute Classification

Multi-task Learning of Cascaded CNN for Facial Attribute Classification.

Multi-Task Network and Optimization for Face Detection and Attribute Analysis

Cross-dataset Face Analysis Based on Multi-Task Learning.

Face Verification with Multi-Task and Multi-Scale Feature Fusion

Facial Action Unit Detection with Multilayer Fused Multi-Task and Multi-Label Deep Learning Network

Multi-Task Deep Neural Network for Joint Face Recognition and Facial Attribute Prediction

A Deep Neural Network-Driven Feature Learning Method for Multi-view Facial Expression Recognition

Attribute-Enhanced Face Recognition with Neural Tensor Fusion Networks.

An Enhanced Facial Expression Recognition Method Based on Multi-task Learning