Multimodal Unsupervised Domain Adaptation for Remote Sensing Image Segmentation

Jiaqi Zou,Zhuohong Li,Fangxiao Lu,Wei He,Hongyan Zhang

DOI: https://doi.org/10.1109/whispers61460.2023.10431324

2023-01-01

Abstract:We propose a multimodal domain-adaptive approach for remote sensing image segmentation, which consists of a multimodal generator network together with a discriminator and adversarial strategy. The multimodal generator employs HR-Net48 as the backbone to construct three specific networks and one shared network, in order to extract domain-specific from individual remote sensing images and shared information from concatenated multimodal images. We adopt AdvEnt as our base scheme for the discriminator and adversarial strategy, leveraging the prior knowledge that source-trained models produce low-entropy (over-confident) predictions for source-like images and high-entropy (under-confident) predictions for target-like images. The discriminator evaluates the entropy map of the generator’s segmentation result to guide the adversarial process. As training progresses, the entropy of the segmentation result reduces for the target domain, achieving domain adaptation from source to target. Finally, the trained generator is applied to unlabeled target domain images to effectively obtain segmentation results.