Abstract:Different from generic image classification, fine-grained classification, such as facial expression classification, in which multiple expressions share inherently similar underlying facial appearances, may show a small difference between facial expression classes. Unlike lab-controlled data, facial expressions from natural scenes have rich forms of the same expression due to the diversity of subjects and the complexity of real-world conditions, and as a result, facial expressions may have large differences among samples within the same class. Moreover, there is little difference between facial expressions, and facial expressions are displayed simultaneously through various facial regions, which require us to encode the feature of multiple key regions, forming high-order interactive information. To address the aforementioned problems, we design an enhanced capsule network based on multi-level feature fusion attention mechanism, which is comprised of four critical components: multi-level feature extraction module (MFEM), multi-level attention module (MAM), multi-level capsule attention fusion module (MCAFM), and reconstruction module (RM). The MFEM collects the low-level, middle-level, and high-level features from the input image, therefore lowering the high-level convolution layer's susceptibility to blurred image and the problem of pose variation. The MAM directs the network's attention to the most significant features in different levels of image features and can assist the network in ignoring blurred, occluded, and irrelevant features and incorporating them into our self-attention center loss function to compress the element distribution in the same class. The MCAFM preserves the attributes of each face region (such as location, size, and direction) by transferring the features into capsules in preparation for the eventual creation of the dynamic routing mechanism, which can resolve the problem of image rotation on FER in the wild. Simultaneously, the capsule features of distinct areas are combined to provide higher-order overall feature information, enhancing the model's capacity to discriminate between different kinds of expressions. The RM reconstructs the image and calculates the difference between the reconstructed image and the original input image. Our model outperforms a large number of current methods on two public datasets, RAF-DB and SFEW.

Robust Face Alignment by Dual-Attentional Spatial-Aware Capsule Networks

SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

End-to-End Spatial Transform Face Detection and Recognition

DSFNet: Dual Space Fusion Network for Occlusion-Robust 3D Dense Face Alignment

Attention-Guided Coarse-to-Fine Network for 2D Face Alignment in the Wild

DSFD: Dual Shot Face Detector

Attention-Mechanism-Based Models for Unconstrained Face Recognition with Mask Occlusion

Feature Correlation-Steered Capsule Network for Object Detection.

Robust Face Alignment Via Adaptive Attention-Based Graph Convolutional Network

Attention-Based Accurate and Robust Facial Landmark Detector

E2-Capsule Neural Networks for Facial Expression Recognition Using AU-Aware Attention

Multistage Model for Robust Face Alignment Using Deep Neural Networks

Dual Attention MobDenseNet(DAMDNet) for Robust 3D Face Alignment

Joint Stacked Hourglass Network and Salient Region Attention Refinement for Robust Face Alignment

Cascaded Structure-Learning Network with Using Adversarial Training for Robust Facial Landmark Detection

A Facial Expression Recognition Method Using Improved Capsule Network Model

Deep Shape Constrained Network for Robust Face Alignment.

TCDCaps: Visual Tracking via Cascaded Dense Capsules.

Dual Sparse Constrained Cascade Regression for Robust Face Alignment.

Stacked Dense U-Nets with Dual Transformers for Robust Face Alignment

Multi-level Feature Fusion Capsule Network with Self-Attention for Facial Expression Recognition