Abstract:Background and objective Unsupervised domain adaptation (UDA) is a powerful approach in tackling domain discrepancies and reducing the burden of laborious and error-prone pixel-level annotations for instance segmentation. However, the domain adaptation strategies utilized in previous instance segmentation models pool all the labeled/detected instances together to train the instance-level GAN discriminator, which neglects the differences among multiple instance categories. Such pooling prevents UDA instance segmentation models from learning categorical correspondence between source and target domains for accurate instance classification; Methods To tackle this challenge, we propose an Instance Segmentation CycleGAN (ISC-GAN) algorithm for UDA multiclass-instance segmentation. We conduct extensive experiments on the multiclass nuclei recognition task to transfer knowledge from hematoxylin and eosin to immunohistochemistry stained pathology images. Specifically, we fuse CycleGAN with Mask R-CNN to learn categorical correspondence with image-level domain adaptation and virtual supervision. Moreover, we utilize Curriculum Learning to separate the learning process into two steps: (1) learning segmentation only on labeled source data, and (2) learning target domain segmentation with paired virtual labels generated by ISC-GAN. The performance was further improved through experiments with other strategies, including Shared Weights, Knowledge Distillation, and Expanded Source Data. Results Comparing to the baseline model or the three UDA instance detection and segmentation models, ISC-GAN illustrates the state-of-the-art performance, with 39.1% average precision and 48.7% average recall. The source codes of ISC-GAN are available at https://github.com/sdw95927/InstanceSegmentation-CycleGAN . Conclusion ISC-GAN adapted knowledge from hematoxylin and eosin to immunohistochemistry stained pathology images, suggesting the potential for reducing the need for large annotated pathological image datasets in deep learning and computer vision tasks.

Reversed domain adaptation for nuclei segmentation-based pathological image classification

Mutual-Complementing Framework for Nuclei Detection and Segmentation in Pathology Image

Domain Adaptation Meets Zero-Shot Learning: an Annotation-Efficient Approach to Multi-Modality Medical Image Segmentation

Learning to Generalize over Subpartitions for Heterogeneity-aware Domain Adaptive Nuclei Segmentation

Unsupervised Instance Segmentation in Microscopy Images Via Panoptic Domain Adaptation and Task Re-Weighting

Unsupervised domain adaptation for nuclei segmentation: Adapting from hematoxylin & eosin stained slides to immunohistochemistry stained slides using a curriculum approach

Global and Local Attentional Feature Alignment for Domain Adaptive Nuclei Detection in Histopathology Images

Black-box Domain Adaptative Cell Segmentation Via Multi-source Distillation

Semi-Supervised Nuclei Detection in Histopathology Images via Location-Aware Adversarial Image Reconstruction

Pulmonary nodules segmentation based on domain adaptation

Cross-domain Nuclei Detection in Histopathology Images Using Graph-based Nuclei Feature Alignment.

Unsupervised Cross-Modality Domain Adaptation of ConvNets for Biomedical Image Segmentations with Adversarial Loss

Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images

Unsupervised Domain Adaptation for the Histopathological Cell Segmentation Through Self-Ensembling

Unsupervised Domain Adaptation Network with Category-Centric Prototype Aligner for Biomedical Image Segmentation

Contrastive and uncertainty-aware nuclei segmentation and classification

NuSegDG: Integration of Heterogeneous Space and Gaussian Kernel for Domain-Generalized Nuclei Segmentation

Instance-Aware Feature Alignment for Cross-Domain Cell Nuclei Detection in Histopathology Images

Nuclei segmentation with point annotations from pathology images via self-supervised learning and co-training

DAWN: Domain-Adaptive Weakly Supervised Nuclei Segmentation via Cross-Task Interactions