Abstract:Effective expression feature representations generated by a triplet-based deep metric learning are highly advantageous for facial expression recognition (FER). The performance of triplet-based deep metric learning is contingent upon identifying the best threshold for triplet loss. Threshold validation, however, is tough and challenging, as the ideal threshold changes among datasets and even across classes within the same dataset. In this paper, we present the multi-threshold deep metric learning technique, which not only avoids the difficult threshold validation but also vastly increases the capacity of triplet loss learning to construct expression feature representations. We find that each threshold of the triplet loss intrinsically determines a distinctive distribution of inter-class variations and corresponds, thus, to a unique expression feature representation. Therefore, rather than selecting a single optimal threshold from a valid threshold range, we thoroughly sample thresholds across the range, allowing the representation characteristics manifested by thresholds within the range to be fully extracted and leveraged for FER. To realize this approach, we partition the embedding layer of the deep metric learning network into a collection of slices and model training these embedding slices as an end-to-end multi-threshold deep metric learning problem. Each embedding slice corresponds to a sample threshold and is learned by enforcing the corresponding triplet loss, yielding a set of distinct expression features, one for each embedding slice. It makes the embedding layer, which is composed of a set of slices, a more informative and discriminative feature, hence enhancing the FER accuracy. Extensive evaluations demonstrate the superior performance of the proposed approach on both posed and spontaneous facial expression datasets.

Adaptive Metric Learning with Deep Neural Networks for Video-Based Facial Expression Recognition

Identity-aware Facial Expression Recognition via Deep Metric Learning based on Synthesized Images

Facial Expression Recognition by Jointly Partial Image and Deep Metric Learning.

Multi-threshold Deep Metric Learning for Facial Expression Recognition

Adaptive discriminative metric learning for facial expression recognition

Dynamic Multi-Channel Metric Network for Joint Pose-Aware and Identity-Invariant Facial Expression Recognition

Learning Expression Features via Deep Residual Attention Networks for Facial Expression Recognition From Video Sequences

Twin attention based multi-task convolutional bidirectional long short term memory for facial expression recognition

A Spatio-Temporal Integrated Model Based on Local and Global Features for Video Expression Recognition

Collaborative Discriminative Multi-Metric Learning for Facial Expression Recognition in Video

Attention-Set based Metric Learning for Video Face Recognition

Video Expression Recognition Method Based on Facial Motion Unit and Temporal Attention

Multi-modal Facial Expression Feature Based on Deep-Neural Networks

Expression-Guided Deep Joint Learning for Facial Expression Recognition

Multi-attention Based Deep Neural Network with Hybrid Features for Dynamic Sequential Facial Expression Recognition

A Triple-Structure Network Model Based Upon MobileNet V1 and Multi-Loss Function for Facial Expression Recognition

Adaptive key-frame selection-based facial expression recognition via multi-cue dynamic features hybrid fusion

3DCNN Backed Conv-LSTM Auto Encoder for Micro Facial Expression Video Recognition

Video facial emotion recognition based on local enhanced motion history image and CNN-CTSLSTM networks

An Enhanced Facial Expression Recognition Method Based on Multi-task Learning

An optimized Capsule-LSTM model for facial expression recognition with video sequences