Abstract:Convolutional neural networks (CNNs) need to replicate feature detectors when modeling spatial information, which reduces their efficiency. The number of replicated feature detectors or labeled training data required for such methods grows exponentially with the dimensionality of the data being used. On the other hand, space-insensitive methods are difficult to encode and express effectively due to the limitation of their rich text structures. In response to the above problems, this paper proposes a capsule network (self-attention capsule network, or SA-CapsNet) with a self-attention mechanism for text classification tasks, wherein the capsule network itself, given the feature with the symmetry hint on two ends, acts as both encoder and decoder. In order to learn long-distance dependent features in sentences and encode text information more efficiently, SA-CapsNet maps the self-attention module to the feature extraction layer of the capsule network, thereby increasing its feature extraction ability and overcoming the limitations of convolutional neural networks. In addition, in this study, in order to improve the accuracy of the model, the capsule was improved by reducing its dimension and an intermediate layer was added, enabling the model to obtain more expressive instantiation features in a given sentence. Finally, experiments were carried out on three general datasets of different sizes, namely the IMDB, MPQA, and MR datasets. The accuracy of the model on these three datasets was 84.72%, 80.31%, and 75.38%, respectively. Furthermore, compared with the benchmark algorithm, the model's performance on these datasets was promising, with an increase in accuracy of 1.08%, 0.39%, and 1.43%, respectively. This study focused on reducing the parameters of the model for various applications, such as edge and mobile applications. The experimental results show that the accuracy is still not apparently decreased by the reduced parameters. The experimental results therefore verify the effective performance of the proposed SA-CapsNet model.

Separable Attention Capsule Network for Signal Classification

Self-Attention Capsule Networks for Object Classification

A Non-Local Capsule Neural Network For Hyperspectral Remote Sensing Image Classification

RS-CapsNet: an Advanced Capsule Network.

Attention-Based Deep Convolutional Capsule Network for Hyperspectral Image Classification

Research on a Capsule Network Text Classification Method with a Self-Attention Mechanism

Hyperspectral Remote Sensing Image Classification Using Deep Convolutional Capsule Network

Capsule Attention Module-Based CapsNet for Hyperspectral Image Classification

A lightweight capsule network via channel-space decoupling and self-attention routing

Spectral–Spatial Hyperspectral Image Classification Using Dual-Channel Capsule Networks

Two-Stream spectral-spatial convolutional capsule network for Hyperspectral image classification

Multiscale Feature Aggregation Capsule Neural Network for Hyperspectral Remote Sensing Image Classification

Multilevel Capsule Weighted Aggregation Network Based on a Decoupled Dynamic Filter for Remote Sensing Scene Classification

TWACapsNet: a capsule network with two-way attention mechanism for speech emotion recognition

Channel-Attention-Based DenseNet Network for Remote Sensing Image Scene Classification

Diverse Capsules Network Combining Multiconvolutional Layers for Remote Sensing Image Scene Classification

Multi-Scale Depthwise Separable Capsule Network for hyperspectral image classification

Using Siamese Capsule Networks for Remote Sensing Scene Classification

CSDS: End-to-End Aerial Scenes Classification With Depthwise Separable Convolution and an Attention Mechanism

A Context-aware Capsule Network for Multi-label Classification

Hyperspectral image classification using multi-level features fusion capsule network with a dense structure