Abstract:Effective expression feature representations generated by a triplet-based deep metric learning are highly advantageous for facial expression recognition (FER). The performance of triplet-based deep metric learning is contingent upon identifying the best threshold for triplet loss. Threshold validation, however, is tough and challenging, as the ideal threshold changes among datasets and even across classes within the same dataset. In this paper, we present the multi-threshold deep metric learning technique, which not only avoids the difficult threshold validation but also vastly increases the capacity of triplet loss learning to construct expression feature representations. We find that each threshold of the triplet loss intrinsically determines a distinctive distribution of inter-class variations and corresponds, thus, to a unique expression feature representation. Therefore, rather than selecting a single optimal threshold from a valid threshold range, we thoroughly sample thresholds across the range, allowing the representation characteristics manifested by thresholds within the range to be fully extracted and leveraged for FER. To realize this approach, we partition the embedding layer of the deep metric learning network into a collection of slices and model training these embedding slices as an end-to-end multi-threshold deep metric learning problem. Each embedding slice corresponds to a sample threshold and is learned by enforcing the corresponding triplet loss, yielding a set of distinct expression features, one for each embedding slice. It makes the embedding layer, which is composed of a set of slices, a more informative and discriminative feature, hence enhancing the FER accuracy. Extensive evaluations demonstrate the superior performance of the proposed approach on both posed and spontaneous facial expression datasets.

Discriminative Deep Multi-Task Learning for Facial Expression Recognition.

Combination of Local Multiple Patterns and Exponential Discriminant Analysis for facial recognition

A Multi-Task Model for Simultaneous Face Identification and Facial Expression Recognition.

Face Recognition Based on LDA and Improved Pairwise-Constrained Multiple Metric Learning Method.

Deep Multi-task Learning for Facial Expression Recognition and Synthesis Based on Selective Feature Sharing

Face Recognition Based on Local Binary Pattern and Improved Pairwise-constrained Multiple Metric Learning

Combining 2D Gabor and Local Binary Pattern for Facial Expression Recognition Using Extreme Learning Machine

Two-Stage Multi-Class Adaboost for Facial Expression Recognition

Multi-feature based automatic facial expression recognition using deep convolutional neural network

Multi-Domain and Multi-Task Learning for Human Action Recognition

Deep Multi-task Multi-label CNN for Effective Facial Attribute Classification

Multi-Instance Hidden Markov Model For Facial Expression Recognition

Scalable Angular Discriminative Deep Metric Learning for Face Recognition

Multi-Task Multi-Modal Self-Supervised Learning for Facial Expression Recognition

Multi-threshold Deep Metric Learning for Facial Expression Recognition

Twin attention based multi-task convolutional bidirectional long short term memory for facial expression recognition

Deep Heterogeneous Multi-Task Metric Learning for Visual Recognition and Retrieval

Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach

Dual-STI: Dual-path spatial-temporal interaction learning for dynamic facial expression recognition

Hybrid Multi-Task Learning for End-To-End Multimodal Emotion Recognition

Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyond