Abstract:Most existing technologies for facial expression recognition utilize off-the-shelf feature extraction methods for classification. In this paper, aiming at learning better features specific for expression representation, we propose to construct a deep architecture, AU-inspired Deep Networks (AUDN), inspired by the psychological theory that expressions can be decomposed into multiple facial Action Units (AUs). To fully exploit this inspiration but avoid detecting AUs, we propose to automatically learn: (1) informative local appearance variation; (2) optimal way to combining local variation and (3) high level representation for final expression recognition. Accordingly, the proposed AUDN is composed of three sequential modules. Firstly, we build a convolutional layer and a max-pooling layer to learn the Micro-Action-Pattern (MAP) representation, which can explicitly depict local appearance variations caused by facial expressions. Secondly, feature grouping is applied to simulate larger receptive fields by combining correlated MAPs adaptively, aiming to generate more abstract mid-level semantics. Finally, a multi-layer learning process is employed in each receptive field respectively to construct group-wise sub-networks for higher-level representations. Experiments on three expression databases CK+, MMI and SFEW demonstrate that, by simply applying linear classifiers on the learned features, our method can achieve state-of-the-art results on all the databases, which validates the effectiveness of AUDN in both lab-controlled and wild environments.

Facial Expression and Action Unit Recognition Augmented by Their Dependencies on Graph Convolutional Networks

Facial Action Units Recognition by De-expression Residue Learning

Data-aware relation learning-based graph convolution neural network for facial action unit recognition

Knowledge Augmented Deep Neural Networks for Joint Facial Expression and Action Unit Recognition.

A Novel Dual-channel Graph Convolutional Neural Network for Facial Action Unit Recognition

Micro-expression Recognition Based on Facial Graph Representation Learning and Facial Action Unit Fusion

AU-aware Graph Convolutional Network for Macroand Micro-expression Spotting.

Facial Action Units Detection Aided by Global-Local Expression Embedding

Facial Action Units Intensity Estimation via Graph Relation Network.

Relation Modeling with Graph Convolutional Networks for Facial Action Unit Detection

Deep Structure Inference Network for Facial Action Unit Recognition

Capture expression-dependent AU relations for expression recognition

An Action Unit Co-Occurrence Constraint 3DCNN Based Action Unit Recognition Approach.

Uncertain Graph Neural Networks for Facial Action Unit Detection

Facial Expression Recognition Via Deep Action Units Graph Network Based on Psychological Mechanism

Dynamic Probabilistic Graph Convolution for Facial Action Unit Intensity Estimation

Multiple Facial Action Unit Recognition Enhanced by Facial Expressions.

Facial Expression Recognition Method Based on a Part-Based Temporal Convolutional Network with a Graph-Structured Representation.

AU-aware Deep Networks for Facial Expression Recognition

AU-inspired Deep Networks for Facial Expression Feature Learning

MER-GCN: Micro Expression Recognition Based on Relation Modeling with Graph Convolutional Network