Abstract:Compared to traditional convolutional neural networks (CNNs), the Capsule network (CapsNet), due to its capsule-based design that aligns better with the principle of human neurons, possesses stronger representation ability by capturing potential spatial structural relationships among different parts of an entity. However, the transformation of neurons into capsules and the iterative routing mechanism result in a considerable computational burden, which is the main drawback of CapsNet. Additionally, the fully connected decoder network exhibits significant reconstruction errors on more complex datasets (e.g., CIFAR-10), which adversely affects the model's classification performance. To this end, this paper proposes a Lightweight Capsule Network (DSA-CapsNet) based on channel-space decoupling and self-attention routing. First, a set of residual blocks is employed to construct the residual extraction layer, where the deep features are decoupled to respectively model the correlations within channel and space, thereby reducing the number of parameters and generating initial capsules simultaneously. Secondly, a self-attention routing algorithm is introduced between capsule layers to effectively handle fewer capsules and then allow for stacking more layers. Lastly, a deconvolution decoder module is used as a better reconstruction method, replacing the fully connected decoder in CapsNet. Through the evaluation of four benchmark datasets, DSA-CapsNet drastically reduces the number of parameters and runtime while exhibiting better classification results. Particularly, on the CIFAR-10 dataset, DSA-CapsNet achieves a 75.38 reduction in parameters compared to the original CapsNet, with a 25.71 increase in classification accuracy.

DeepCaps: Going Deeper With Capsule Networks

Ddrm-Capsnet: Capsule Network Based On Deep Dynamic Routing Mechanism For Complex Data

DE-CapsNet: A Diverse Enhanced Capsule Network with Disperse Dynamic Routing

Training Deep Capsule Networks with Residual Connections

A lightweight capsule network via channel-space decoupling and self-attention routing

A novel capsule network based on deep routing and residual learning

Capsule Network Performance on Complex Data

Efficient-CapsNet: capsule network with self-attention routing

Pushing the Limits of Capsule Networks

A novel dense capsule network based on dense capsule layers

Capsule Networks With Residual Pose Routing

R-CapsNet: an Improvement of Capsule Network for More Complex Data

Deep multi-prototype capsule networks

RS-CapsNet: an Advanced Capsule Network.

Path Capsule Networks

CapsNets Algorithm

FSC-CapsNet: Fractionally-Strided Convolutional Capsule Network for Complex Data.

Hybrid Gromov-Wasserstein Embedding for Capsule Learning

MC-CapsNet : Low-level Pooling and High-level Fusion of Multi-columns Capsule Network

An Improvement for Capsule Networks using Depthwise Separable Convolution

Cv-CapsNet: Complex-Valued Capsule Network