Abstract:Facial micro-expression (ME), consisting of uncontrollable muscle movements in faces, is an important clue for revealing real people's feelings. Due to the short duration and low intensity, the salient feature representation learning is the main challenge for robust facial ME recognition. To acquire the diverse and spatial relation representation, this paper proposes a simple and yet distinctive micro-expression recognition model based on multiscale convolutional fusion and multi-channel capsule network (MCFMCN). Firstly, the apex frame in a ME clip, located by computing the pixel difference between frames, is filtered by the optical flow transformation. Secondly, a multiscale fusion module is introduced to capture diverse ME related details. Then, to further explore the subtle spatial relations between parts in the ME faces, the multi-channel capsule network is designed to improve the feature representation performance of the traditional single channel capsule network. Finally, the entire ME recognition model is trained and verified on three benchmarks (CASMEII, SAMM, and SMIC) using the associated standard evaluation protocols: unweighted average recall rate (UAR) and unweighted F1 score (UF1). ME recognition experiments indicate that our method based on MCFMCN can improve the UAR (from 75.79% to 83.58%) and UF1(from79.37% to 87.06%) in comparison with the traditional capsule network. Extensive experimental results show the performance of proposed ME recognition is superior to that of works based on pervious single channel capsule network or other state-of-the-art CNN models, which validates the finding that combination of multi-scale analysis and multi-channel capsule network is feasible and effective to improve the ME recognition performance.

Structure Representation with Adaptive and Compact Facial Graph for Micro-Expression Recognition

Geometric Graph Representation with Learnable Graph Structure and Adaptive AU Constraint for Micro-Expression Recognition

Adaptive Graph Attention Network with Temporal Fusion for Micro-Expressions Recognition

MFDAN: Multi-level Flow-Driven Attention Network for Micro-Expression Recognition

PASTFNet: a paralleled attention spatio-temporal fusion network for micro-expression recognition

Facial micro-expression recognition based on motion magnification network and graph attention mechanism

Micro-Expression Recognition with Layered Relations and More Input Frames

The heterogeneous ensemble of deep forest and deep neural networks for micro-expressions recognition

A Novel Micro-Expression Recognition Approach Using Attention-Based Magnification-Adaptive Networks

Recognizing Spontaneous Micro-Expression Using a Three-Stream Convolutional Neural Network.

Multi-channel Capsule Network for Micro-expression Recognition with Multiscale Fusion

DFFCN: Dual Flow Fusion Convolutional Network for Micro Expression Recognition

ME-PLAN: A deep prototypical learning with local attention network for dynamic micro-expression recognition

Dual-stream network with cross-layer attention and similarity constraint for micro-expression recognition

Feature Representation Learning with Adaptive Displacement Generation and Transformer Fusion for Micro-Expression Recognition

Micro-Expression Recognition by Motion Feature Extraction based on Pre-training

Diverse Local Facial Behaviors Learning from Enhanced Expression Flow for Microexpression Recognition

SAANet: Siamese Action-Units Attention Network for Improving Dynamic Facial Expression Recognition

Inceptr: micro-expression recognition integrating inception-CBAM and vision transformer

A Boost in Revealing Subtle Facial Expressions: A Consolidated Eulerian Framework

Highly Effective End-to-End Single-to-Multichannel Feature Fusion and Ensemble Classification to Decode Emotional Secretes from Small-Scale Spontaneous Facial Micro-Expressions