Abstract:Land use and land cover maps provide fundamental information that has been used in different types of studies, ranging from public health to carbon cycling. However, the existing remote sensing image classification methods thus far suffer from the insufficient usage of multiple modalities, underconsideration of prior domain knowledge, and poor performance on minority classes. To alleviate these problems, we propose a novel domain knowledge-guided deep collaborative fusion network (DKDFN) with performance boosting for minority categories for land cover classification. More specifically, the DKDFN adopts a multihead encoder and a multibranch decoder structure. The architecture of the encoder probablizes sufficient mining of complementary information from multiple modalities, which are Sentinel-2, Sentinel-1, and SRTM Digital Elevation Data (SRTM) in our case. The multibranch decoder enables land cover classification in a multitask learning setup, performing semantic segmentation and reconstructing multimodal remote sensing indices, which are selected as representatives of domain knowledge. This design incorporates domain knowledge in an effective end-to-end manner. The training stage of our DKDFN is supervised by our proposed asymmetry loss function (ALF), which boosts performance on nearly all categories, especially the categories with a low frequency of occurrence. Ablation studies of the network suggest that our design logic is worth testing in any network with an encoder-decoder structure. The study is conducted in Hunan, China and is verified using a self-labeled multimodal unitemporal remote sensing image dataset. The comparative experiments between DKDFN and 6 state-of-the-art models (U-Net, SegNet, PSPNet, DeepLab, HRNet, MP-ResNet) testify to the superiority of our method and suggest its potential to be applied more widely to map land cover in other geographical areas given the availability of Sentinel-2, Sentinel-1, and SRTM data. The dataset can be downloaded by https://github.com/LauraChow/HunanMultimodalDataset.

Robust Land Cover Classification with Local-Global Information Decoupling to Address Remote Sensing Anomalous Data

A lightweight and stochastic depth residual attention network for remote sensing scene classification

Semantic Representation Fusion-Based Network for Robust Land Cover Classification in Foggy Conditions.

Superpixel-Based Long-Range Dependent Network for High-Resolution Remote-Sensing Image Classification

Robust Semisupervised Land-Use Classification Using Remote Sensing Data With Weak Labels

DO-Net: Dual-Output Network for Land Cover Classification From Optical Remote Sensing Images

Classification of Very-High-Spatial-Resolution Aerial Images Based on Multiscale Features with Limited Semantic Information

How Well Do Deep Learning-Based Methods for Land Cover Classification and Object Detection Perform on High Resolution Remote Sensing Imagery?

DKDFN: Domain Knowledge-Guided deep collaborative fusion network for multimodal unitemporal remote sensing land cover classification

Local feature acquisition and global context understanding network for very high-resolution land cover classification

Urban Land Cover Classification With Missing Data Modalities Using Deep Convolutional Neural Networks

A Land Cover Classification Method for High-Resolution Remote Sensing Images Based on NDVI Deep Learning Fusion Network

Assisted learning for land use classification: The important role of semantic correlation between heterogeneous images

Multigranularity Decoupling Network With Pseudolabel Selection for Remote Sensing Image Scene Classification

Cognitive Land Cover Mapping: A Three-Layer Deep Learning Architecture for Remote Sensing Data Classification

Localization, balance and affinity: a stronger multifaceted collaborative salient object detector in remote sensing images

Cross-Resolution Land Cover Classification Using Outdated Products and Transformers

Land-cover classification with high-resolution remote sensing images using transferable deep models

A new strategy based on multi-source remote sensing data for improving the accuracy of land use/cover change classification

A deep inverse convolutional neural network-based semantic classification method for land cover remote sensing images

Advancing Remote Sensing: A Unified Deep Learning Approach with Pretrained and Custom Architectures for High-Precision Classification