Abstract:To achieve effective deep fusion features for improving the classification accuracy of hyperspectral remote sensing images (HRSIs), a pixel frequency spectrum feature is presented and introduced to convolutional neural networks (CNNs). Firstly, the fast Fourier transform is performed on each spectral pixel to obtain the amplitude spectrum, i.e., the pixel frequency spectrum feature. Then, the obtained pixel frequency spectrum is combined with the spectral pixel to form a mixed feature, i.e., spectral and frequency spectrum mixed feature (SFMF). Several multi-branch CNNs fed with pixel frequency spectrum, SFMF, spectral pixel, and spatial features are designed for extracting deep fusion features. A pre-learning strategy, i.e., basic single branch CNNs are used to pre-learn the weights of a multi-branch CNN, is also presented for improving the network convergence speed and avoiding the network from getting into a locally optimal solution to a certain extent. And after reducing the dimensionality of SFMF by principal component analysis (PCA), a 3-dimensionality (3-D) CNN is also designed to further extract the joint spatial-SFMF feature. The experimental results of three real HRSIs show that adding the presented frequency spectrum feature into CNNs can achieve better recognition results, which in turn proves that the presented multi-branch CNNs can obtain the deep fusion features with more discriminant information.

Design of Intelligent Spectral Selection and Multispectral Imaging Fusion Algorithm Based on Deep Neural Network Model

Fusion of Low-Illuminance Visible and Near-Infrared Images Based on Convolutional Neural Networks

Hyperspectral and Multispectral Image Fusion Based on Deep Attention Network.

Aerial Scene Classification Via Multilevel Fusion Based on Deep Convolutional Neural Networks.

FUSING DIFFERENT LEVELS OF DEEP FEATURES BY DEEP STACKED NEURAL NETWORK FOR HYPERSPECTRAL IMAGES

A Spatial–Spectral Dual-Optimization Model-Driven Deep Network for Hyperspectral and Multispectral Image Fusion

FusionCNN: a Remote Sensing Image Fusion Algorithm Based on Deep Convolutional Neural Networks

Deep Fusion of Remote Sensing Data for Accurate Classification.

Spatial-Spectral Fusion by Combining Deep Learning and Variation Model

Hyperspectral and Multispectral Image Fusion by Deep Neural Network in a Self-Supervised Manner

An Efficient Cross-Modality Self-Calibrated Network for Hyperspectral and Multispectral Image Fusion

DAFCNN: A Dual-Channel Feature Extraction and Attention Feature Fusion Convolution Neural Network for SAR Image and MS Image Fusion

Hyperspectral and Multispectral Data Fusion with 1D-Convolution on Spectrum.

Deep Multi-Feature Fusion Network for Remote Sensing Images

Deep CNN-based Hyperspectral Image Classification Using Discriminative Multiple Spatial-Spectral Feature Fusion

Spatial–Spectral Fusion by Combining Deep Learning and Variational Model

Hyperspectral Remote Sensing Images Deep Feature Extraction Based on Mixed Feature and Convolutional Neural Networks

Deep Fusion of Hyperspectral and LiDAR Data for Thematic Classification.

A New Deep Neural Network for Optical and SAR Image Fusion

A Novel Deep Feature Fusion Network for Remote Sensing Scene Classification