Abstract:Recently, the capsule network and its enhanced version named convolutional capsule network (Conv-CapsNet) were applied to hyperspectral image (HSI) classification and delivered outstanding performance, attributing to the enriched capabilities of capsules for feature expression and complex spatial knowledge excavation. However, restricted by the locality of convolutions in primary feature extraction, the rich spectral-spatial diagnostic information found in HSIs were insufficiently utilized, which degrades classification performance to a certain degree. In addition, most capsule networks involve a complex structure with many parameters, which are haunted by a common bottleneck of limited available training samples. To tackle these challenges, this paper presents a novel two-stream spectral-spatial convolutional capsule network (TSCCN) for HSIs to boost accurate classification with small samples. Based on the advanced convolutional capsule concept, a two-stream Conv-CapsNet architecture is developed that reduces trainable parameters and mitigates overfitting when samples are insufficient. Specifically, a 1D Conv-CapsNet and a 2D Conv-CapsNet sharing similar structures are developed as accompanying streams to fully excavate favorable features from the continuous spectra and raw spectral-spatial cubes, with the separate features fused to generate joint deep features for classification. To overcome the locality of primary convolutions, an efficient structural information mining module (SIM) is designed for each stream, to learn complementary cross-channel and cross-kernel inter-dependencies for subsequent capsule learning and further improve classification performance. Experiments on three well-known HSI datasets demonstrate the superiority of the proposed algorithm over nine representative state-of-the-art deep classifiers and its efficacy under small sample conditions, where all the methods were implemented in the Tensor-Flow framework and repeated for 10 times.

Scale-invariant Convolutional Capsule Network

Deformable Capsules for Object Detection

Capsule Dynamic Network-Based Object Detection Algorithm

A novel dense capsule network based on dense capsule layers

1D-Convolutional Capsule Network for Hyperspectral Image Classification

Two-Stream spectral-spatial convolutional capsule network for Hyperspectral image classification

Patch-Based Capsule Network for Complex Images

Feature Amplification Capsule Network for Complex Images.

RS-CapsNet: an Advanced Capsule Network.

RGB-D salient object detection via convolutional capsule network based on feature extraction and integration

RGB-D Salient Object Detection Based on Feature Extraction and Integration Convolutional Capsule Network

Modified Capsule Network For Object Classification

Hyperspectral Remote Sensing Image Classification Using Deep Convolutional Capsule Network

Orthogonal Capsule Networks With Positional Information Preservation and Lightweight Feature Learning

A lightweight capsule network via channel-space decoupling and self-attention routing

Pushing the Limits of Capsule Networks

Hyperspectral image classification using multi-level features fusion capsule network with a dense structure

Capsule Network Performance on Complex Data

Subspace Capsule Network

A Vision Enhancement and Feature Fusion Multiscale Detection Network

CapsNets Algorithm