Abstract:Micro-expression, different from ordinary facial expressions, is an involuntary, spontaneous, and subtle facial movement that reveals true emotions which people intend to conceal. As it usually occurs within a fraction of a second (less than 1/2 second) with a low action intensity, capturing micro-expressions among facial movements in a video is difficult. Moreover, when a micro-expression recognition system works in cold-start conditions, it has to recognize novel classes of micro-expressions in a new scenario, suffering from the lack of sufficient labeled samples. Inconsistency in micro-expression labeling criteria makes it difficult to use existing labeled datasets in other scenarios. To tackle the challenges, we present a micro-expression recognizer, which on one hand leverages the knowledge of facial action units (AU) to enhance facial representations, and on the other hand performs cross-domain few-shot learning to transfer knowledge acquired from other domains with different data labeling protocols and feature distribution to overcome the scarcity of labeled samples in the cold-starting scenario. In particular, we draw inspirations from the correlation between micro-expression and facial action units (AUs), and design an action unit module, aiming to extract subtle AU-related features from videos. We then fuse AU-related features and general features extracted by optical-flow facial images. Through fine-tuning, we transfer knowledge from datasets in different domains to the target domain. The experimental results on two datasets show that: (1) the proposed recognizer can effectively learn to recognize new categories of micro-expressions in different domains with a very few labeled samples with the UF1 score of 0.544 on CASME dataset, outperforming the state-of-the-art methods by 0.089; (2) the performance of the recognizer is more competitive when it distinguishes micro-expression videos of more categories; and (3) the action unit module enables to improve the recognition performance by 0.072 and 0.047 on CASME and SMIC, respectively.

Multiple source domain adaptation in micro-expression recognition

Pose-robust personalized facial expression recognition through unsupervised multi-source domain adaptation

Multiscale Attention-Based Subdomain Dynamic Adaptation for Cross-Domain Scene Classification

Cross-Domain Few-Shot Micro-Expression Recognition Incorporating Action Units.

Toward Bridging Microexpressions From Different Domains

Multi-Representation Joint Dynamic Domain Adaptation Network for Cross-Database Facial Expression Recognition

Domain Regeneration for Cross-Database Micro-Expression Recognition

Cross-Database Micro-Expression Recognition Exploiting Intradomain Structure

Pose-aware Adversarial Domain Adaptation for Personalized Facial Expression Recognition

Micro-Expression Recognition Enhanced by Macro-Expression from Spatial-Temporal Domain.

Multi-Source and Multi-Representation Adaptation for Cross-Domain Electroencephalography Emotion Recognition

Micro-Attention for Micro-Expression recognition

Decoupling facial motion features and identity features for micro-expression recognition

Unsupervised Domain Adaptation for Face Anti-Spoofing

Multi-Stream Facial Adaptive Network for Expression Recognition from a Single Image

Micro-expression recognition based on multi-scale 3D residual convolutional neural network

Multi-source joint domain adaptation for cross-subject and cross-session emotion recognition from electroencephalography

Deep Margin-Sensitive Representation Learning for Cross-Domain Facial Expression Recognition

Seeking Salient Facial Regions for Cross-Database Micro-Expression Recognition

Three-Stream Temporal-Shift Attention Network Based on Self-Knowledge Distillation for Micro-Expression Recognition