Abstract:Recently, the capsule network and its enhanced version named convolutional capsule network (Conv-CapsNet) were applied to hyperspectral image (HSI) classification and delivered outstanding performance, attributing to the enriched capabilities of capsules for feature expression and complex spatial knowledge excavation. However, restricted by the locality of convolutions in primary feature extraction, the rich spectral-spatial diagnostic information found in HSIs were insufficiently utilized, which degrades classification performance to a certain degree. In addition, most capsule networks involve a complex structure with many parameters, which are haunted by a common bottleneck of limited available training samples. To tackle these challenges, this paper presents a novel two-stream spectral-spatial convolutional capsule network (TSCCN) for HSIs to boost accurate classification with small samples. Based on the advanced convolutional capsule concept, a two-stream Conv-CapsNet architecture is developed that reduces trainable parameters and mitigates overfitting when samples are insufficient. Specifically, a 1D Conv-CapsNet and a 2D Conv-CapsNet sharing similar structures are developed as accompanying streams to fully excavate favorable features from the continuous spectra and raw spectral-spatial cubes, with the separate features fused to generate joint deep features for classification. To overcome the locality of primary convolutions, an efficient structural information mining module (SIM) is designed for each stream, to learn complementary cross-channel and cross-kernel inter-dependencies for subsequent capsule learning and further improve classification performance. Experiments on three well-known HSI datasets demonstrate the superiority of the proposed algorithm over nine representative state-of-the-art deep classifiers and its efficacy under small sample conditions, where all the methods were implemented in the Tensor-Flow framework and repeated for 10 times.

A Non-Local Capsule Neural Network For Hyperspectral Remote Sensing Image Classification

A Hyperspectral Image Classification Method Based on the Nonlocal Attention Mechanism of a Multiscale Convolutional Neural Network.

Hyperspectral Remote Sensing Image Classification Using Deep Convolutional Capsule Network

Multiscale Feature Aggregation Capsule Neural Network for Hyperspectral Remote Sensing Image Classification

Attention-Based Deep Convolutional Capsule Network for Hyperspectral Image Classification

Capsule Attention Module-Based CapsNet for Hyperspectral Image Classification

Spectral–spatial Classification of Hyperspectral Remote Sensing Image Based on Capsule Network

Hyperspectral Image Classification with CapsNet and Markov Random Fields

Spnet: A Spectral Patching Network For End-To-End Hyperspectral Image Classification

Spectral–Spatial Hyperspectral Image Classification Using Dual-Channel Capsule Networks

Robust Capsule Network Based on Maximum Correntropy Criterion for Hyperspectral Image Classification.

1D-Convolutional Capsule Network for Hyperspectral Image Classification

Prototypical Network With Residual Capsule for Few-Shot Hyperspectral Image Classification

Deep Convolutional Capsule Network for Hyperspectral Image Spectral and Spectral-Spatial Classification.

Hyperspectral image classification using multi-level features fusion capsule network with a dense structure

Two-Stream spectral-spatial convolutional capsule network for Hyperspectral image classification

Multi-Scale Depthwise Separable Capsule Network for hyperspectral image classification

Generative Adversarial Capsule Network With ConvLSTM for Hyperspectral Image Classification

Hyperspectral Image Classification Using CapsNet With Well-Initialized Shallow Layers

Capsule Attention Network for Hyperspectral Image Classification

Dual-Channel Capsule Generation Adversarial Network for Hyperspectral Image Classification.