Abstract:Micro-expression, different from ordinary facial expressions, is an involuntary, spontaneous, and subtle facial movement that reveals true emotions which people intend to conceal. As it usually occurs within a fraction of a second (less than 1/2 second) with a low action intensity, capturing micro-expressions among facial movements in a video is difficult. Moreover, when a micro-expression recognition system works in cold-start conditions, it has to recognize novel classes of micro-expressions in a new scenario, suffering from the lack of sufficient labeled samples. Inconsistency in micro-expression labeling criteria makes it difficult to use existing labeled datasets in other scenarios. To tackle the challenges, we present a micro-expression recognizer, which on one hand leverages the knowledge of facial action units (AU) to enhance facial representations, and on the other hand performs cross-domain few-shot learning to transfer knowledge acquired from other domains with different data labeling protocols and feature distribution to overcome the scarcity of labeled samples in the cold-starting scenario. In particular, we draw inspirations from the correlation between micro-expression and facial action units (AUs), and design an action unit module, aiming to extract subtle AU-related features from videos. We then fuse AU-related features and general features extracted by optical-flow facial images. Through fine-tuning, we transfer knowledge from datasets in different domains to the target domain. The experimental results on two datasets show that: (1) the proposed recognizer can effectively learn to recognize new categories of micro-expressions in different domains with a very few labeled samples with the UF1 score of 0.544 on CASME dataset, outperforming the state-of-the-art methods by 0.089; (2) the performance of the recognizer is more competitive when it distinguishes micro-expression videos of more categories; and (3) the action unit module enables to improve the recognition performance by 0.072 and 0.047 on CASME and SMIC, respectively.

Facial Action Units Recognition: A Comparative Study

A COMPARATIVE STUDY OF HMMS AND DBNS APPLIED TO FACIAL ACTION UNITS RECOGNITION

Fully Automatic Upper Facial Action Recognition.

3D Facial Action Units Recognition for Emotional Expression

Facial Expression Recognition Based on Facial Action Unit

Fully automatic recognition of the temporal phases of facial actions

Relative Facial Action Unit Detection

Leveraging Previous Facial Action Units Knowledge for Emotion Recognition on Faces

Multiple Facial Action Unit Recognition Enhanced by Facial Expressions.

A Robust Real-Time System for Multi-Intensity AU Detection in Facial Expression Recognition

Action unit analysis enhanced facial expression recognition by deep neural network evolution

Analysis, Interpretation, and Recognition of Facial Action Units and Expressions Using Neuro-Fuzzy Modeling

Facial Action Units Detection Aided by Global-Local Expression Embedding

Facial action units detection using temporal context and feature reassignment

AU-Expression Knowledge Constrained Representation Learning for Facial Expression Recognition

Cross-Domain Few-Shot Micro-Expression Recognition Incorporating Action Units.

Objective Classes for Micro-Facial Expression Recognition

SAANet: Siamese Action-Units Attention Network for Improving Dynamic Facial Expression Recognition

Fully automatic 3D facial expression recognition using a region-based approach

Expression-assisted Facial Action Unit Recognition under Incomplete AU Annotation.

Learning facial expression-aware global-to-local representation for robust action unit detection