Abstract:Recently, the capsule network and its enhanced version named convolutional capsule network (Conv-CapsNet) were applied to hyperspectral image (HSI) classification and delivered outstanding performance, attributing to the enriched capabilities of capsules for feature expression and complex spatial knowledge excavation. However, restricted by the locality of convolutions in primary feature extraction, the rich spectral-spatial diagnostic information found in HSIs were insufficiently utilized, which degrades classification performance to a certain degree. In addition, most capsule networks involve a complex structure with many parameters, which are haunted by a common bottleneck of limited available training samples. To tackle these challenges, this paper presents a novel two-stream spectral-spatial convolutional capsule network (TSCCN) for HSIs to boost accurate classification with small samples. Based on the advanced convolutional capsule concept, a two-stream Conv-CapsNet architecture is developed that reduces trainable parameters and mitigates overfitting when samples are insufficient. Specifically, a 1D Conv-CapsNet and a 2D Conv-CapsNet sharing similar structures are developed as accompanying streams to fully excavate favorable features from the continuous spectra and raw spectral-spatial cubes, with the separate features fused to generate joint deep features for classification. To overcome the locality of primary convolutions, an efficient structural information mining module (SIM) is designed for each stream, to learn complementary cross-channel and cross-kernel inter-dependencies for subsequent capsule learning and further improve classification performance. Experiments on three well-known HSI datasets demonstrate the superiority of the proposed algorithm over nine representative state-of-the-art deep classifiers and its efficacy under small sample conditions, where all the methods were implemented in the Tensor-Flow framework and repeated for 10 times.

Spectral–Spatial Hyperspectral Image Classification Using Dual-Channel Capsule Networks

Spectral–spatial Classification of Hyperspectral Remote Sensing Image Based on Capsule Network

Hyperspectral Remote Sensing Image Classification Using Deep Convolutional Capsule Network

Hyperspectral Image Classification with Capsule Network Using Limited Training Samples.

Hyperspectral Image Classification with CapsNet and Markov Random Fields

Deep Convolutional Capsule Network for Hyperspectral Image Spectral and Spectral-Spatial Classification.

Multiscale Feature Aggregation Capsule Neural Network for Hyperspectral Remote Sensing Image Classification

Capsule Attention Module-Based CapsNet for Hyperspectral Image Classification

1D-Convolutional Capsule Network for Hyperspectral Image Classification

Attention-Based Deep Convolutional Capsule Network for Hyperspectral Image Classification

Dual-Channel Capsule Generation Adversarial Network for Hyperspectral Image Classification.

Two-Stream spectral-spatial convolutional capsule network for Hyperspectral image classification

Multi-scale Convolutional Capsule Network for Hyperspectral Image Classification

Hyperspectral image classification using multi-level features fusion capsule network with a dense structure

Robust Capsule Network Based on Maximum Correntropy Criterion for Hyperspectral Image Classification.

Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network

A Non-Local Capsule Neural Network For Hyperspectral Remote Sensing Image Classification

Hyperspectral Image Classification Based on Dual-Channel Network with Spectral-Spatial Fusion Attention

Multi-Scale Depthwise Separable Capsule Network for hyperspectral image classification

Faster Multiscale Capsule Network with Octave Convolution for Hyperspectral Image Classification

Prototypical Network With Residual Capsule for Few-Shot Hyperspectral Image Classification