Data-Efficient Domain Adaptation for Semantic Segmentation of Aerial Imagery Using Generative Adversarial Networks

Bilel Benjdira,Adel Ammar,Anis Koubaa,Kais Ouni

DOI: https://doi.org/10.3390/app10031092

2020-02-06

Applied Sciences

Abstract:Despite the significant advances noted in semantic segmentation of aerial imagery, a considerable limitation is blocking its adoption in real cases. If we test a segmentation model on a new area that is not included in its initial training set, accuracy will decrease remarkably. This is caused by the domain shift between the new targeted domain and the source domain used to train the model. In this paper, we addressed this challenge and proposed a new algorithm that uses Generative Adversarial Networks (GAN) architecture to minimize the domain shift and increase the ability of the model to work on new targeted domains. The proposed GAN architecture contains two GAN networks. The first GAN network converts the chosen image from the target domain into a semantic label. The second GAN network converts this generated semantic label into an image that belongs to the source domain but conserves the semantic map of the target image. This resulting image will be used by the semantic segmentation model to generate a better semantic label of the first chosen image. Our algorithm is tested on the ISPRS semantic segmentation dataset and improved the global accuracy by a margin up to 24% when passing from Potsdam domain to Vaihingen domain. This margin can be increased by addition of other labeled data from the target domain. To minimize the cost of supervision in the translation process, we proposed a methodology to use these labeled data efficiently.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the domain shift between different domains, that is, when testing the segmentation model in new areas not included in the initial training set, the accuracy will drop significantly. This phenomenon is caused by the differences between the target domain and the source domain used to train the model. Specifically, the paper focuses on how to improve the adaptability and accuracy of the semantic segmentation model in new domains without a large increase in annotated data. To meet this challenge, the author proposes a new algorithm that utilizes the Generative Adversarial Network (GAN) architecture to minimize domain shift and enhance the model's adaptability to new target domains. The core idea of this algorithm is to use two GAN networks in cascade to convert the images in the target domain into the style of the source domain while keeping their semantic information unchanged, so that the pre - trained segmentation model can better process the images in the target domain. The main contributions of this method are as follows: 1. Significantly alleviates the domain shift problem in semantic segmentation tasks, and the accuracy of the model can be further improved by efficiently adding a small amount of annotated data in the target domain. 2. The effectiveness of the method has been verified through experiments on the ISPRS semantic annotation data set, especially in cross - domain semantic segmentation between the Vaihingen and Potsdam data sets, with the overall accuracy rate increased by up to 24%. 3. Introduces GAN as an effective solution for analyzing aerial images, demonstrating its potential in reducing domain shift.

Data-Efficient Domain Adaptation for Semantic Segmentation of Aerial Imagery Using Generative Adversarial Networks

SPGAN-DA: Semantic-Preserved Generative Adversarial Network for Domain Adaptive Remote Sensing Image Semantic Segmentation

Exploiting Semantics in Adversarial Training for Image-Level Domain Adaptation

StandardGAN: Multi-source Domain Adaptation for Semantic Segmentation of Very High Resolution Satellite Images by Data Standardization

Semantic-Aware Generative Adversarial Nets for Unsupervised Domain Adaptation in Chest X-ray Segmentation

Semantically Adaptive Image-to-image Translation for Domain Adaptation of Semantic Segmentation

Conditional Generative Adversarial Networks for Data Augmentation and Adaptation in Remotely Sensed Imagery

Self-Ensembling With GAN-Based Data Augmentation for Domain Adaptation in Semantic Segmentation

Semi-supervised semantic segmentation based on Generative Adversarial Networks for remote sensing images

IMPROVING SEMANTIC SEGMENTATION OF HIGH-RESOLUTION REMOTE SENSING IMAGES USING WASSERSTEIN GENERATIVE ADVERSARIAL NETWORK

Evaluating generative adversarial networks based image-level domain transfer for multi-source remote sensing image segmentation and object detection

Domain Adaptation for Remote Sensing Image Semantic Segmentation: An Integrated Approach of Contrastive Learning and Adversarial Learning

Generate To Adapt: Aligning Domains using Generative Adversarial Networks

Unsupervised domain adaptation for remote sensing semantic segmentation with the 2D discrete wavelet transform

GenGMM: Generalized Gaussian-Mixture-based Domain Adaptation Model for Semantic Segmentation

Decomposition-Based Unsupervised Domain Adaptation for Remote Sensing Image Semantic Segmentation

Depth-Assisted ResiDualGAN for Cross-Domain Aerial Images Semantic Segmentation

Variational Domain Adaptation Driven Semantic Segmentation of Urban Scenes

A Mutual Information Domain Adaptation Network for Remotely Sensed Semantic Segmentation

IterDANet: Iterative Intra-Domain Adaptation for Semantic Segmentation of Remote Sensing Images

Fully Convolutional Adaptation Networks for Semantic Segmentation