Abstract:Although deep learning has made semantic segmentation of very-high-resolution (VHR) remote sensing (RS) images practical and efficient, its large-scale application is still limited. Given the diversity of imaging sensors, acquisition conditions, and regional styles, a deep learning network well-trained on one source domain dataset often suffers from drastic performance drops when applied to other target domain datasets. Thus, we propose a novel end-to-end mutual information domain adaptation network (MIDANet) that can shift between semantic segmentation domains by integrating multitask learning in the convolutional neural networks within an entropy adversarial learning (EAL) framework. Through the joint learning of semantic segmentation and elevation estimation, the features extracted by MIDANet can concentrate more on the elevation clues while dropping the domain-variant information (i.e., texture, spectral information). First, one encoder is applied to excavate general semantic features. Two decoders that share the same architecture are used to perform pixel-level classification and digital surface model (DSM) regression. Second, feature interaction modules (FIMs) and a mutual information attention unit (MIAU) are designed to mine the latent relationships between the two tasks and enhance their feature representations. Finally, a final MIDANet is obtained for semantic segmentation that does not require any semantic segmentation labels in the target domain after the adversarial learning of the classification entropy at the output level. Extensive comparative experiments and ablation studies were conducted on the International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam and Vaihingen test datasets. The results show that MIDANet outperforms other state-of-the-art domain adaptation (DA) methods in both evaluation metrics and visual assessment.

MultiDAN: Unsupervised, Multistage, Multisource and Multitarget Domain Adaptation for Semantic Segmentation of Remote Sensing Images

IterDANet: Iterative Intra-Domain Adaptation for Semantic Segmentation of Remote Sensing Images

MDANet: Unsupervised, Mixed-Domain Adaptation for Semantic Segmentation of Remote Sensing Images.

Unsupervised Domain Adaptation for Remote Sensing Image Semantic Segmentation Using Region and Category Adaptive Domain Discriminator

Multi-Target Unsupervised Domain Adaptation for Semantic Segmentation without External Data

A Mutual Information Domain Adaptation Network for Remotely Sensed Semantic Segmentation

Self-training and Multi-level Adversarial Network for Domain Adaptive Remote Sensing Image Segmentation

Optimal transport-based domain adaptation for semantic segmentation of remote sensing images

Multi-Scale Depth-Aware Unsupervised Domain Adaption in Semantic Segmentation.

Unsupervised Domain Adaptation Augmented by Mutually Boosted Attention for Semantic Segmentation of VHR Remote Sensing Images

MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation

A Multilevel-Guided Curriculum Domain Adaptation Approach to Semantic Segmentation for High-Resolution Remote Sensing Images.

A Fine-Grained Unsupervised Domain Adaptation Framework for Semantic Segmentation of Remote Sensing Images

DA&MTSS: an End-to-End Remote Sensing Image Domain Adaptive Semantic Segmentation Framework Combining Data Augmentation and Mobile Threshold Self-supervision

Unsupervised Domain Adaptation Semantic Segmentation for Remote-Sensing Images Via Covariance Attention

CMT: Cross Mean Teacher Unsupervised Domain Adaptation for VHR Image Semantic Segmentation

Multi-source Domain Adaptation for Semantic Segmentation

DASRSNet: Multitask Domain Adaptation for Super-Resolution-Aided Semantic Segmentation of Remote Sensing Images

Mdalu: Multi-Source Domain Adaptation and Label Unification with Partial Datasets

A Multi-Grained Unsupervised Domain Adaptation Approach for Semantic Segmentation

Unsupervised Mixed Multi-Target Domain Adaptation for Remote Sensing Images Classification