Abstract:Appearances and qualities of remote sensing images are affected by different atmospheric conditions, quality of sensors, and radiometric calibrations. This heavily challenges the generalization ability of a deep learning or other machine learning model: the performance of a model pretrained on a source remote sensing data set can significantly decrease when applied to a different target data set. The popular generative adversarial networks (GANs) can realize style or appearance transfer between a source and target data sets, which may boost the performance of a deep learning model through generating new target images similar to source samples. In this study, we comprehensively evaluate the performance of GAN-based image-level transfer methods on convolutional neural network (CNN) based image processing models that are trained on one dataset and tested on another one. Firstly, we designed the framework for the evaluation process. The framework consists of two main parts, the GAN-based image-level domain adaptation, which transfers a target image to a new image with similar probability distribution of source image space, and the CNN-based image processing tasks, which are used to test the effects of GAN-based domain adaptation. Second, the domain adaptation is implemented with two mainstream GAN methods for style transfer, the CycleGAN and the AgGAN. The image processing contains two major tasks, segmentation and object detection. The former and the latter are designed based on the widely applied U-Net and Faster R-CNN, respectively. Finally, three experiments, associated with three datasets, are designed to cover different application cases, a change detection case where temporal data is collected from the same scene, a two-city case where images are collected from different regions and a two-sensor case where images are obtained from aerial and satellite platforms respectively. Results revealed that, the GAN-based image transfer can significantly boost the performance of the segmentation model in the change detection case, however, it did not surpass conventional methods; in the other two cases, the GAN-based methods obtained worse results. In object detection, almost all the methods failed to boost the performance of the Faster R-CNN and the GAN-based methods performed the worst.

ST-GAN: A Swin Transformer-Based Generative Adversarial Network for Unsupervised Domain Adaptation of Cross-Modality Cardiac Segmentation.

ISGAN: Unsupervised Domain Adaptation with Improved Symmetric GAN for Cross-Modality Multi-organ Segmentation

Bidirectional cross-modality unsupervised domain adaptation using generative adversarial networks for cardiac image segmentation

TSP-UDANet: Two-Stage Progressive Unsupervised Domain Adaptation Network for Automated Cross-Modality Cardiac Segmentation

10.1109/ICIPMC62364.2024.10586680 Unsupervised Domain Adaptation for Cross-Modality Cardiac Image Segmentation Based on Contrastive Image Synthesis

Unsupervised Domain Adaptation for Cross-Modality Cardiac Image Segmentation Based on Contrastive Image Synthesis

MSGAN: Multi-Stage Generative Adversarial Networks for Cross-Modality Domain Adaptation.

Automated cardiac segmentation of cross-modal medical images using unsupervised multi-domain adaptation and spatial neural attention structure

The Domain Shift Problem of Medical Image Segmentation and Vendor-Adaptation by Unet-GAN

Swin transformer-based GAN for multi-modal medical image translation

ST-Unet: Swin Transformer boosted U-Net with Cross-Layer Feature Enhancement for medical image segmentation

U-shaped GAN for Semi-Supervised Learning and Unsupervised Domain Adaptation in High Resolution Chest Radiograph Segmentation

SWCGAN: Generative Adversarial Network Combining Swin Transformer and CNN for Remote Sensing Image Super-Resolution

Semantic Segmentation by Improved Generative Adversarial Networks

Semantic-Aware Generative Adversarial Nets for Unsupervised Domain Adaptation in Chest X-ray Segmentation

Disentangle Domain Features for Cross-Modality Cardiac Image Segmentation.

Applying Conditional Generative Adversarial Networks for Imaging Diagnosis

Evaluating generative adversarial networks based image-level domain transfer for multi-source remote sensing image segmentation and object detection

Unsupervised Domain Adaptation Augmented by Mutually Boosted Attention for Semantic Segmentation of VHR Remote Sensing Images

Multimodal CT Image Synthesis Using Unsupervised Deep Generative Adversarial Networks for Stroke Lesion Segmentation

Unsupervised Cross-modality Adaptation via Dual Structural-Oriented Guidance for 3D Medical Image Segmentation