Abstract:Recently, the capsule network and its enhanced version named convolutional capsule network (Conv-CapsNet) were applied to hyperspectral image (HSI) classification and delivered outstanding performance, attributing to the enriched capabilities of capsules for feature expression and complex spatial knowledge excavation. However, restricted by the locality of convolutions in primary feature extraction, the rich spectral-spatial diagnostic information found in HSIs were insufficiently utilized, which degrades classification performance to a certain degree. In addition, most capsule networks involve a complex structure with many parameters, which are haunted by a common bottleneck of limited available training samples. To tackle these challenges, this paper presents a novel two-stream spectral-spatial convolutional capsule network (TSCCN) for HSIs to boost accurate classification with small samples. Based on the advanced convolutional capsule concept, a two-stream Conv-CapsNet architecture is developed that reduces trainable parameters and mitigates overfitting when samples are insufficient. Specifically, a 1D Conv-CapsNet and a 2D Conv-CapsNet sharing similar structures are developed as accompanying streams to fully excavate favorable features from the continuous spectra and raw spectral-spatial cubes, with the separate features fused to generate joint deep features for classification. To overcome the locality of primary convolutions, an efficient structural information mining module (SIM) is designed for each stream, to learn complementary cross-channel and cross-kernel inter-dependencies for subsequent capsule learning and further improve classification performance. Experiments on three well-known HSI datasets demonstrate the superiority of the proposed algorithm over nine representative state-of-the-art deep classifiers and its efficacy under small sample conditions, where all the methods were implemented in the Tensor-Flow framework and repeated for 10 times.

Hyperspectral image classification using multi-level features fusion capsule network with a dense structure

Multiscale 3-D-2-D Mixed CNN and Lightweight Attention-Free Transformer for Hyperspectral and LiDAR Classification

Spectral–Spatial Hyperspectral Image Classification Using Dual-Channel Capsule Networks

1D-Convolutional Capsule Network for Hyperspectral Image Classification

Hyperspectral Remote Sensing Image Classification Using Deep Convolutional Capsule Network

Capsule Attention Module-Based CapsNet for Hyperspectral Image Classification

Multi-layer feature fusion for hyperspectral image classification

A Non-Local Capsule Neural Network For Hyperspectral Remote Sensing Image Classification

Two-Stream spectral-spatial convolutional capsule network for Hyperspectral image classification

Multi-Scale Depthwise Separable Capsule Network for hyperspectral image classification

Attention-Based Deep Convolutional Capsule Network for Hyperspectral Image Classification

Aerial Scene Classification Via Multilevel Fusion Based on Deep Convolutional Neural Networks.

Hyperspectral image classification based on a novel Lush multi-layer feature fusion bias network

Multi-Level Feature Extraction Networks for Hyperspectral Image Classification

Hyperspectral Image Classification Based on Multilevel Joint Feature Extraction Network

MSFF: A Multi-Scale Feature Fusion Convolutional Neural Network for Hyperspectral Image Classification

Hyperspectral Image Classification Based on Interactive Transformer and CNN With Multilevel Feature Fusion Network

A Novel 2D-3D CNN with Spectral-Spatial Multi-Scale Feature Fusion for Hyperspectral Image Classification

Multiscale Information Fusion for Hyperspectral Image Classification Based on Hybrid 2D-3D CNN

A novel dense capsule network based on dense capsule layers

A Feature Embedding Network with Multiscale Attention for Hyperspectral Image Classification