Abstract:Convolutional neural networks (CNNs) need to replicate feature detectors when modeling spatial information, which reduces their efficiency. The number of replicated feature detectors or labeled training data required for such methods grows exponentially with the dimensionality of the data being used. On the other hand, space-insensitive methods are difficult to encode and express effectively due to the limitation of their rich text structures. In response to the above problems, this paper proposes a capsule network (self-attention capsule network, or SA-CapsNet) with a self-attention mechanism for text classification tasks, wherein the capsule network itself, given the feature with the symmetry hint on two ends, acts as both encoder and decoder. In order to learn long-distance dependent features in sentences and encode text information more efficiently, SA-CapsNet maps the self-attention module to the feature extraction layer of the capsule network, thereby increasing its feature extraction ability and overcoming the limitations of convolutional neural networks. In addition, in this study, in order to improve the accuracy of the model, the capsule was improved by reducing its dimension and an intermediate layer was added, enabling the model to obtain more expressive instantiation features in a given sentence. Finally, experiments were carried out on three general datasets of different sizes, namely the IMDB, MPQA, and MR datasets. The accuracy of the model on these three datasets was 84.72%, 80.31%, and 75.38%, respectively. Furthermore, compared with the benchmark algorithm, the model's performance on these datasets was promising, with an increase in accuracy of 1.08%, 0.39%, and 1.43%, respectively. This study focused on reducing the parameters of the model for various applications, such as edge and mobile applications. The experimental results show that the accuracy is still not apparently decreased by the reduced parameters. The experimental results therefore verify the effective performance of the proposed SA-CapsNet model.

An Improved Capsule Network Based On Newly Reconstructed Network And The Method Of Sharing Parameters

Modified Capsule Network For Object Classification

R-CapsNet: an Improvement of Capsule Network for More Complex Data

Reducing the dilution: An analysis of the information sensitiveness of capsule network with a practical improvement method

An Improved Capsule Network Based on Capsule Filter Routing

Patch-Based Capsule Network for Complex Images

A lightweight capsule network via channel-space decoupling and self-attention routing

Multi-Lane Capsule Network for Classifying Images With Complex Background

Reducing the dilution: An analysis of the information sensitiveness of capsule network with a practical solution.

An Improvement for Capsule Networks using Depthwise Separable Convolution

A novel capsule network based on deep routing and residual learning

Efficient-CapsNet: capsule network with self-attention routing

Research on a Capsule Network Text Classification Method with a Self-Attention Mechanism

Capsule Network Performance on Complex Data

Fast Convergent Capsule Network with Applications in MNIST.

The Analysis Between Traditional Convolution Neural Network and CapsuleNet

A Novel Capsule Network Based on Multi-Order Descartes Extension Transformation

RS-CapsNet: an Advanced Capsule Network.

Ddrm-Capsnet: Capsule Network Based On Deep Dynamic Routing Mechanism For Complex Data

Modified Algorithm of Capsule Network for Classifying Small Sample Image

Path Capsule Networks