Abstract:Convolutional neural networks (CNNs) need to replicate feature detectors when modeling spatial information, which reduces their efficiency. The number of replicated feature detectors or labeled training data required for such methods grows exponentially with the dimensionality of the data being used. On the other hand, space-insensitive methods are difficult to encode and express effectively due to the limitation of their rich text structures. In response to the above problems, this paper proposes a capsule network (self-attention capsule network, or SA-CapsNet) with a self-attention mechanism for text classification tasks, wherein the capsule network itself, given the feature with the symmetry hint on two ends, acts as both encoder and decoder. In order to learn long-distance dependent features in sentences and encode text information more efficiently, SA-CapsNet maps the self-attention module to the feature extraction layer of the capsule network, thereby increasing its feature extraction ability and overcoming the limitations of convolutional neural networks. In addition, in this study, in order to improve the accuracy of the model, the capsule was improved by reducing its dimension and an intermediate layer was added, enabling the model to obtain more expressive instantiation features in a given sentence. Finally, experiments were carried out on three general datasets of different sizes, namely the IMDB, MPQA, and MR datasets. The accuracy of the model on these three datasets was 84.72%, 80.31%, and 75.38%, respectively. Furthermore, compared with the benchmark algorithm, the model's performance on these datasets was promising, with an increase in accuracy of 1.08%, 0.39%, and 1.43%, respectively. This study focused on reducing the parameters of the model for various applications, such as edge and mobile applications. The experimental results show that the accuracy is still not apparently decreased by the reduced parameters. The experimental results therefore verify the effective performance of the proposed SA-CapsNet model.

SA-CapsGAN: Using Capsule Networks with Embedded Self-Attention for Generative Adversarial Network

E-CapsGan: Generative Adversarial Network Using Capsule Network As Feature Encoder

CapsuleGAN: Generative Adversarial Capsule Network

Capsule GAN Using Capsule Network for Generator Architecture

Research on a Capsule Network Text Classification Method with a Self-Attention Mechanism

A lightweight capsule network via channel-space decoupling and self-attention routing

Adaptive Capsule Network

Subspace Capsule Network

A novel dense capsule network based on dense capsule layers

RS-CapsNet: an Advanced Capsule Network.

DeepCaps: Going Deeper With Capsule Networks

Self-Attention Capsule Networks for Object Classification

Pushing the Limits of Capsule Networks

Hybrid Gromov-Wasserstein Embedding for Capsule Learning

FSC-CapsNet: Fractionally-Strided Convolutional Capsule Network for Complex Data.

Comparing Generative Adversarial Network Techniques for Image Creation and Modification

CBAM-GAN: Generative Adversarial Networks Based on Convolutional Block Attention Module

P-CapsNets: a General Form of Convolutional Neural Networks

Ddrm-Capsnet: Capsule Network Based On Deep Dynamic Routing Mechanism For Complex Data

Capsule Network Performance on Complex Data

Generalized Capsule Networks with Trainable Routing Procedure