Abstract:Land cover classification (LCC) is an important application in remote sensing data interpretation and invariably faces big intra-class variance and sample imbalance in remote sensing images. The optical image is obtained by satellites capturing the spectral information of the Earth’s surface, and the synthetic aperture radar (SAR) image is produced by the satellite actively transmitting and receiving the electromagnetic wave signals reflected from land covers. Because of the limitations of the optical image, a single modality (optical image) might be disturbed by external conditions, especially complex weather. Using heterogeneous SAR and optical images for LCC can reduce the negative impact caused by single-modal data damage, and multi-modal data can also be used as supplementary information to enhance classification accuracy. However, general LCC methods mainly focus on remote sensing data of a single modality without fully considering the multi-modalities of land covers. Therefore, we propose a dual-stream deep high-resolution network (DDHRNet) to deeply integrate SAR and optical data at the feature level in every branch. The network can effectively exploit the complementary information in heterogeneous images. It improves classification performance and achieves significant improvements in the classification of clouded images. A multi-modal squeeze-and-excitation (MSE) module is also utilized to fuse the features. Compared with the ordinary methods, MSE modules can lead to an improvement of about 1% to 5% in overall accuracy (OA), Kappa coefficients, and mean intersection over union (mIoU). Besides, in order to evaluate our method, we describe in detail the preprocessing process of Gaofen-2 (GF2) and Gaofen-3 (GF3) data before they are used in the LCC task. The experiments show that the proposed method performs well compared with other excellent segmentation methods and obtains the best performance on heterogeneous images from GF2 and GF3. The code and datasets are available at: https://github.com/XD-MG/DDHRNet.

Bidirectional Grid Fusion Network for Accurate Land Cover Classification of High-Resolution Remote Sensing Images

Aerial Scene Classification Via Multilevel Fusion Based on Deep Convolutional Neural Networks.

GCFnet: Global Collaborative Fusion Network for Multispectral and Panchromatic Image Classification

DGFNet: Dual Gate Fusion Network for Land Cover Classification in Very High-Resolution Images

Novel Cross-Resolution Feature-Level Fusion for Joint Classification of Multispectral and Panchromatic Remote Sensing Images

Dual Attention Feature Fusion and Adaptive Context for Accurate Segmentation of Very High-Resolution Remote Sensing Images

Deep Feature Fusion for High-Resolution Aerial Scene Classification

CFNet: A Cross Fusion Network for Joint Land Cover Classification Using Optical and SAR Images

Multi-Modality and Multi-Scale Attention Fusion Network for Land Cover Classification from VHR Remote Sensing Images

AMFFNet: Attention-Guided Multi-Level Feature Fusion Network for Land Cover Classification of Remote Sensing Images

A Novel Adaptive Hybrid Fusion Network for Multiresolution Remote Sensing Images Classification

SABNet:Self-Attention Bilateral Network for Land Cover Classification

Hierarchical Attention and Bilinear Fusion for Remote Sensing Image Scene Classification

A Hybrid Deep Convolutional Neural Network for Accurate Land Cover Classification.

Local feature acquisition and global context understanding network for very high-resolution land cover classification

Multibranch Feature Fusion Network with Self- and Cross-Guided Attention for Hyperspectral and LiDAR Classification

An encoder–decoder network for land cover classification using a fusion of aerial images and photogrammetric point clouds

A Spatial-Channel Progressive Fusion ResNet for Remote Sensing Classification.

Collaborative Attention-Based Heterogeneous Gated Fusion Network for Land Cover Classification

A Dual-Stream High Resolution Network: Deep Fusion of GF-2 and GF-3 Data for Land Cover Classification

Very High Resolution Images Classification by Fusing Deep Convolutional Neural Networks