Abstract:Micro-expression, different from ordinary facial expressions, is an involuntary, spontaneous, and subtle facial movement that reveals true emotions which people intend to conceal. As it usually occurs within a fraction of a second (less than 1/2 second) with a low action intensity, capturing micro-expressions among facial movements in a video is difficult. Moreover, when a micro-expression recognition system works in cold-start conditions, it has to recognize novel classes of micro-expressions in a new scenario, suffering from the lack of sufficient labeled samples. Inconsistency in micro-expression labeling criteria makes it difficult to use existing labeled datasets in other scenarios. To tackle the challenges, we present a micro-expression recognizer, which on one hand leverages the knowledge of facial action units (AU) to enhance facial representations, and on the other hand performs cross-domain few-shot learning to transfer knowledge acquired from other domains with different data labeling protocols and feature distribution to overcome the scarcity of labeled samples in the cold-starting scenario. In particular, we draw inspirations from the correlation between micro-expression and facial action units (AUs), and design an action unit module, aiming to extract subtle AU-related features from videos. We then fuse AU-related features and general features extracted by optical-flow facial images. Through fine-tuning, we transfer knowledge from datasets in different domains to the target domain. The experimental results on two datasets show that: (1) the proposed recognizer can effectively learn to recognize new categories of micro-expressions in different domains with a very few labeled samples with the UF1 score of 0.544 on CASME dataset, outperforming the state-of-the-art methods by 0.089; (2) the performance of the recognizer is more competitive when it distinguishes micro-expression videos of more categories; and (3) the action unit module enables to improve the recognition performance by 0.072 and 0.047 on CASME and SMIC, respectively.

An Action Unit Co-Occurrence Constraint 3DCNN Based Action Unit Recognition Approach.

Multiple Facial Action Unit Recognition Enhanced by Facial Expressions.

Action unit analysis enhanced facial expression recognition by deep neural network evolution

Facial Action Unit Recognition and Inference for Facial Expression Analysis.

A Novel Dual-channel Graph Convolutional Neural Network for Facial Action Unit Recognition

Capture expression-dependent AU relations for expression recognition

3D Facial Action Units Recognition for Emotional Expression

Facial Action Units Recognition: A Comparative Study

Expression-assisted Facial Action Unit Recognition under Incomplete AU Annotation.

A comprehensive survey on automatic facial action unit analysis

Learning facial expression-aware global-to-local representation for robust action unit detection

AU-Expression Knowledge Constrained Representation Learning for Facial Expression Recognition

Knowledge Augmented Deep Neural Networks for Joint Facial Expression and Action Unit Recognition.

Facial Action Units Detection Aided by Global-Local Expression Embedding

Deep Structure Inference Network for Facial Action Unit Recognition

Data-aware relation learning-based graph convolution neural network for facial action unit recognition

Cross-Domain Few-Shot Micro-Expression Recognition Incorporating Action Units.

Multi-scale Dynamic and Hierarchical Relationship Modeling for Facial Action Units Recognition

Facial Emotion and Action Unit Recognition Based on Bayesian Network.

Facial Expression Recognition Based on Facial Action Unit

Facial action units detection using temporal context and feature reassignment