Abstract:Convolutional neural networks (CNNs) need to replicate feature detectors when modeling spatial information, which reduces their efficiency. The number of replicated feature detectors or labeled training data required for such methods grows exponentially with the dimensionality of the data being used. On the other hand, space-insensitive methods are difficult to encode and express effectively due to the limitation of their rich text structures. In response to the above problems, this paper proposes a capsule network (self-attention capsule network, or SA-CapsNet) with a self-attention mechanism for text classification tasks, wherein the capsule network itself, given the feature with the symmetry hint on two ends, acts as both encoder and decoder. In order to learn long-distance dependent features in sentences and encode text information more efficiently, SA-CapsNet maps the self-attention module to the feature extraction layer of the capsule network, thereby increasing its feature extraction ability and overcoming the limitations of convolutional neural networks. In addition, in this study, in order to improve the accuracy of the model, the capsule was improved by reducing its dimension and an intermediate layer was added, enabling the model to obtain more expressive instantiation features in a given sentence. Finally, experiments were carried out on three general datasets of different sizes, namely the IMDB, MPQA, and MR datasets. The accuracy of the model on these three datasets was 84.72%, 80.31%, and 75.38%, respectively. Furthermore, compared with the benchmark algorithm, the model's performance on these datasets was promising, with an increase in accuracy of 1.08%, 0.39%, and 1.43%, respectively. This study focused on reducing the parameters of the model for various applications, such as edge and mobile applications. The experimental results show that the accuracy is still not apparently decreased by the reduced parameters. The experimental results therefore verify the effective performance of the proposed SA-CapsNet model.

TWACapsNet: a capsule network with two-way attention mechanism for speech emotion recognition

Two-dimensional discrete feature based spatial attention CapsNet For sEMG signal recognition

Attention-Enhanced Connectionist Temporal Classification for Discrete Speech Emotion Recognition

Speech Emotion Recognition Using Sequential Capsule Networks

Self-attention Transfer Networks for Speech Emotion Recognition

CTL-MTNet: A Novel CapsNet and Transfer Learning-Based Mixed Task Net for the Single-Corpus and Cross-Corpus Speech Emotion Recognition

Speech Emotion Recognition Based on Convolutional Neural Network with Attention-Based Bidirectional Long Short-Term Memory Network and Multi-Task Learning

Combining a parallel 2D CNN with a self-attention Dilated Residual Network for CTC-based discrete speech emotion recognition

CENN: Capsule-enhanced neural network with innovative metrics for robust speech emotion recognition

Speech Emotion Recognition by Combining a Unified First-Order Attention Network with Data Balance

Speech Emotion Recognition Via CNN-Transformer and Multidimensional Attention Mechanism

Novel Dual-Channel Long Short-Term Memory Compressed Capsule Networks for Emotion Recognition

TC-Net: A Transformer Capsule Network for EEG-based emotion recognition

Dilated Residual Network with Multi-head Self-attention for Speech Emotion Recognition

MSAC: Multiple Speech Attribute Control Method for Reliable Speech Emotion Recognition

Cross-Corpus Speech Emotion Recognition Based on Hybrid Neural Networks

Pre-trained Deep Convolution Neural Network Model With Attention for Speech Emotion Recognition

Ms-senet: Enhancing Speech Emotion Recognition Through Multi-scale Feature Fusion With Squeeze-and-excitation Blocks

RES-CapsNet: an improved capsule network for micro-expression recognition

E2-Capsule Neural Networks for Facial Expression Recognition Using AU-Aware Attention

Research on a Capsule Network Text Classification Method with a Self-Attention Mechanism