Abstract:Introduction:Micro-expressions are facial muscle movements that hide genuine emotions. In response to the challenge of micro-expression low-intensity, recent studies have attempted to locate localized areas of facial muscle movement. However, this ignores the feature redundancy caused by the inaccurate locating of the regions of interest.Methods:This paper proposes a novel multi-scale fusion visual attention network (MFVAN), which learns multi-scale local attention weights to mask regions of redundancy features. Specifically, this model extracts the multi-scale features of the apex frame in the micro-expression video clips by convolutional neural networks. The attention mechanism focuses on the weights of local region features in the multi-scale feature maps. Then, we mask operate redundancy regions in multi-scale features and fuse local features with high attention weights for micro-expression recognition. The self-supervision and transfer learning reduce the influence of individual identity attributes and increase the robustness of multi-scale feature maps. Finally, the multi-scale classification loss, mask loss, and removing individual identity attributes loss joint to optimize the model.Results:The proposed MFVAN method is evaluated on SMIC, CASME II, SAMM, and 3DB-Combined datasets that achieve state-of-the-art performance. The experimental results show that focusing on local at the multi-scale contributes to micro-expression recognition.Discussion:This paper proposed MFVAN model is the first to combine image generation with visual attention mechanisms to solve the combination challenge problem of individual identity attribute interference and low-intensity facial muscle movements. Meanwhile, the MFVAN model reveal the impact of individual attributes on the localization of local ROIs. The experimental results show that a multi-scale fusion visual attention network contributes to micro-expression recognition.

Multi-View Facial Expression Recognition with Multi-View Facial Expression Light Weight Network

A New Deep Learning Method for Multi-label Facial Expression Recognition Based on Local Constraint Features

A Deep Neural Network-Driven Feature Learning Method for Multi-view Facial Expression Recognition

A Lightweight Attention-based Deep Network via Multi-Scale Feature Fusion for Multi-View Facial Expression Recognition

Faceknow: Facial Expression Recognition by a Global-Local Network with a Sub-Images-Related Contextual Attention Mechanism

Multi-View Facial Expression Recognition Based O N Multitask Learning and Generative Adversarial Network.

Joint spatial and scale attention network for multi-view facial expression recognition

Multi-scale Fusion Visual Attention Network for Facial Micro-Expression Recognition.

A discriminative multiscale feature extraction network for facial expression recognition in the wild

A Multi-Head Convolution Network with Attention Consistency for Facial Expression Recognition

Dynamic Multi-Channel Metric Network for Joint Pose-Aware and Identity-Invariant Facial Expression Recognition

Micro-expression recognition based on multi-scale 3D residual convolutional neural network

A Multi-stream Convolutional Neural Network for Micro-expression Recognition Using Optical Flow and EVM

Locality-constrained Linear Coding Based Bi-Layer Model for Multi-View Facial Expression Recognition

Multi-View Facial Expression Recognition Based on Group Sparse Reduced-Rank Regression

Facial Expression Recognition Using Visible, IR, and MSX Images by Early and Late Fusion of Deep Learning Models

Lightweight Facial Expression Recognition Based on Multi-scale Dense Convolutional Neural Network

Multi-view Facial Expression Recognition Based on Fusing Low-level and Mid-level Features

A Triple-Structure Network Model Based Upon MobileNet V1 and Multi-Loss Function for Facial Expression Recognition

Expression Recognition Based on Multi-feature Fusion of Peak-neutral Differences

An Enhanced Facial Expression Recognition Method Based on Multi-task Learning