Abstract:Micro-expression, different from ordinary facial expressions, is an involuntary, spontaneous, and subtle facial movement that reveals true emotions which people intend to conceal. As it usually occurs within a fraction of a second (less than 1/2 second) with a low action intensity, capturing micro-expressions among facial movements in a video is difficult. Moreover, when a micro-expression recognition system works in cold-start conditions, it has to recognize novel classes of micro-expressions in a new scenario, suffering from the lack of sufficient labeled samples. Inconsistency in micro-expression labeling criteria makes it difficult to use existing labeled datasets in other scenarios. To tackle the challenges, we present a micro-expression recognizer, which on one hand leverages the knowledge of facial action units (AU) to enhance facial representations, and on the other hand performs cross-domain few-shot learning to transfer knowledge acquired from other domains with different data labeling protocols and feature distribution to overcome the scarcity of labeled samples in the cold-starting scenario. In particular, we draw inspirations from the correlation between micro-expression and facial action units (AUs), and design an action unit module, aiming to extract subtle AU-related features from videos. We then fuse AU-related features and general features extracted by optical-flow facial images. Through fine-tuning, we transfer knowledge from datasets in different domains to the target domain. The experimental results on two datasets show that: (1) the proposed recognizer can effectively learn to recognize new categories of micro-expressions in different domains with a very few labeled samples with the UF1 score of 0.544 on CASME dataset, outperforming the state-of-the-art methods by 0.089; (2) the performance of the recognizer is more competitive when it distinguishes micro-expression videos of more categories; and (3) the action unit module enables to improve the recognition performance by 0.072 and 0.047 on CASME and SMIC, respectively.

Recognizing micro actions in videos by learning multi-layer local features

Deep Local Video Feature for Action Recognition

Learning and Distillating the Internal Relationship of Motion Features in Action Recognition.

Action Recognition By Learning Deep Multi-Granular Spatio-Temporal Video Representation

Human Action Recognition From Digital Videos Based on Deep Learning.

Human Action Recognition Using Deep Learning Methods.

Improved Convolutional 3D Networks for Micro-Movements Recognition

Multilayer deep features with multiple kernel learning for action recognition

Facial Micro-Expression Recognition Based on Multi-Scale Temporal and Spatial Features

Fusing $${\mathcal {R}}$$R Features and Local Features with Context-Aware Kernels for Action Recognition

Cross-Domain Few-Shot Micro-Expression Recognition Incorporating Action Units.

Learning Hierarchical Video Representation for Action Recognition

Action Recognition with Joint Attention on Multi-Level Deep Features

Local Feature Analysis for real-time Action Recognition.

Local-aware spatio-temporal attention network with multi-stage feature fusion for human action recognition

Deep Learning-Based Human Action Recognition in Videos

Micro-Expression Recognition by Aggregating Local Spatio-Temporal Patterns.

Multi-scale residual network model combined with Global Average Pooling for action recognition

Action Recognition by Hierarchical Mid-level Action Elements

Human Action Recognition Based on Hierarchical Multi-Scale Adaptive Conv-Long Short-Term Memory Network

Action Recognition in Videos through a Transfer-Learning-Based Technique