Abstract:Facial Expression Recognition (FER) is one of the most important topics in the domain of computer vision and pattern recognition, and it has attracted increasing attention for its scientific challenges and application potentials. In this article, we propose a novel and effective approach to FER using multi-model two-dimensional (2D) and 3D videos, which encodes both static and dynamic clues by scattering convolution network. First, a shape-based detection method is introduced to locate the start and the end of an expression in videos; segment its onset, apex, and offset states; and sample the important frames for emotion analysis. Second, the frames in Apex of 2D videos are represented by scattering, conveying static texture details. Those of 3D videos are processed in a similar way, but to highlight static shape details, several geometric maps in terms of multiple order differential quantities, i.e., Normal Maps and Shape Index Maps, are generated as the input of scattering, instead of original smooth facial surfaces. Third, the average of neighboring samples centred at each key texture frame or shape map in Onset is computed, and the scattering features extracted from all the average samples of 2D and 3D videos are then concatenated to capture dynamic texture and shape cues, respectively. Finally, Multiple Kernel Learning is adopted to combine the features in the 2D and 3D modalities and compute similarities to predict the expression label. Thanks to the scattering descriptor, the proposed approach not only encodes distinct local texture and shape variations of different expressions as by several milestone operators, such as SIFT, HOG, and so on, but also captures subtle information hidden in high frequencies in both channels, which is quite crucial to better distinguish expressions that are easily confused. The validation is conducted on the BU-4DFE and BP-4D databa ses, and the accuracies reached are very competitive, indicating its competency for this issue.

DrFER: Learning Disentangled Representations for 3D Facial Expression Recognition

DR-FER: Discriminative and Robust Representation Learning for Facial Expression Recognition

Progressive Self-supervised Representation Learning for 3D Facial Expression Recognition

2D+3D Facial Expression Recognition via Discriminative Dynamic Range Enhancement and Multi-Scale Learning

3D-FERNet: A Facial Expression Recognition Network utilizing 3D information

Cross-domain Facial Expression Recognition Via Disentangling Identity Representation

Disentangling Identity and Pose for Facial Expression Recognition

Automatic 4D Facial Expression Recognition via Collaborative Cross-domain Dynamic Image Network.

D³Net: Dual-Branch Disturbance Disentangling Network for Facial Expression Recognition.

Automatic 3D Facial Expression Recognition Using Geometric Scattering Representation

Towards Reading Beyond Faces for Sparsity-aware 3D/4D Affect Recognition

Rethinking the Learning Paradigm for Dynamic Facial Expression Recognition

Deep Representation of Facial Geometric and Photometric Attributes for Automatic 3D Facial Expression Recognition

Combining 2D Gabor and Local Binary Pattern for Facial Expression Recognition Using Extreme Learning Machine

Landmarks-assisted Collaborative Deep Framework for Automatic 4D Facial Expression Recognition.

Facial Expression Recognition with Two-Branch Disentangled Generative Adversarial Network.

An Optimal Transportation-Based Recognition Algorithm for 3D Facial Expressions

Facial Expression Recognition with Geometric Scattering on 3D Point Clouds

Discriminative Attention-based Convolutional Neural Network for 3D Facial Expression Recognition.

3D facial expression recognition using delta faces

Texture and Geometry Scattering Representation-Based Facial Expression Recognition in 2D+3D Videos