Abstract: This paper considers the problem of image set-based face verification and identification. Unlike traditional single sample (an image or a video) setting, this situation assumes the availability of a set of heterogeneous collection of orderless images and videos. The samples can be taken at different check points, different identity documents $etc$. The importance of each image is usually considered either equal or based on a quality assessment of that image independent of other images and/or videos in that image set. How to model the relationship of orderless images within a set remains a challenge. We address this problem by formulating it as a Markov Decision Process (MDP) in a latent space. Specifically, we first propose a dependency-aware attention control (DAC) network, which uses actor-critic reinforcement learning for attention decision of each image to exploit the correlations among the unordered images. An off-policy experience replay is introduced to speed up the learning process. Moreover, the DAC is combined with a temporal model for videos using divide and conquer strategies. We also introduce a pose-guided representation (PGR) scheme that can further boost the performance at extreme poses. We propose a parameter-free PGR without the need for training as well as a novel metric learning-based PGR for pose alignment without the need for pose detection in testing stage. Extensive evaluations on IJB-A/B/C, YTF, Celebrity-1000 datasets demonstrate that our method outperforms many state-of-art approaches on the set-based as well as video-based face recognition databases.

Multitask Learning Strategy with Pseudo-Labeling: Face Recognition, Facial Landmark Detection, and Head Pose Estimation

A Real-Time Multi-Task Learning System for Joint Detection of Face, Facial Landmark and Head Pose

Toward High Quality Facial Representation Learning

Face Recognition Based on LDA and Improved Pairwise-Constrained Multiple Metric Learning Method.

Towards Mask-robust Face Recognition.

TRFH: Towards Real-Time Face Detection and Head Pose Estimation.

Dual Learning for Joint Facial Landmark Detection and Action Unit Recognition

MOS: A Low Latency and Lightweight Framework for Face Detection, Landmark Localization, and Head Pose Estimation

Multi-task pose-invariant face recognition.

Dynamic Facial Expression Recognition with Pseudo‐label Guided Multi‐modal Pre‐training

Teacher Supervises Students How to Learn From Partially Labeled Images for Facial Landmark Detection

Facial Landmark Detection Via Self-Adaption Model and Multi-Task Feature Learning

Attention Control with Metric Learning Alignment for Image Set-based Recognition

Multi-task Face Analyses Through Adversarial Learning.

A Robust Facial Landmark Detector with Mixed Loss

Face Detection With End-To-End Integration Of A Convnet And A 3d Model

Cross-dataset Face Analysis Based on Multi-Task Learning.

Deep Multi-task Multi-label CNN for Effective Facial Attribute Classification

A New Joint Training Method for Facial Expression Recognition with Inconsistently Annotated and Imbalanced Data

Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks

Recognizing Partial Biometric Patterns.