Abstract:Background and objective Unsupervised domain adaptation (UDA) is a powerful approach in tackling domain discrepancies and reducing the burden of laborious and error-prone pixel-level annotations for instance segmentation. However, the domain adaptation strategies utilized in previous instance segmentation models pool all the labeled/detected instances together to train the instance-level GAN discriminator, which neglects the differences among multiple instance categories. Such pooling prevents UDA instance segmentation models from learning categorical correspondence between source and target domains for accurate instance classification; Methods To tackle this challenge, we propose an Instance Segmentation CycleGAN (ISC-GAN) algorithm for UDA multiclass-instance segmentation. We conduct extensive experiments on the multiclass nuclei recognition task to transfer knowledge from hematoxylin and eosin to immunohistochemistry stained pathology images. Specifically, we fuse CycleGAN with Mask R-CNN to learn categorical correspondence with image-level domain adaptation and virtual supervision. Moreover, we utilize Curriculum Learning to separate the learning process into two steps: (1) learning segmentation only on labeled source data, and (2) learning target domain segmentation with paired virtual labels generated by ISC-GAN. The performance was further improved through experiments with other strategies, including Shared Weights, Knowledge Distillation, and Expanded Source Data. Results Comparing to the baseline model or the three UDA instance detection and segmentation models, ISC-GAN illustrates the state-of-the-art performance, with 39.1% average precision and 48.7% average recall. The source codes of ISC-GAN are available at https://github.com/sdw95927/InstanceSegmentation-CycleGAN . Conclusion ISC-GAN adapted knowledge from hematoxylin and eosin to immunohistochemistry stained pathology images, suggesting the potential for reducing the need for large annotated pathological image datasets in deep learning and computer vision tasks.

Global and Local Attentional Feature Alignment for Domain Adaptive Nuclei Detection in Histopathology Images

Instance-Aware Feature Alignment for Cross-Domain Cell Nuclei Detection in Histopathology Images

Cross-domain Nuclei Detection in Histopathology Images Using Graph-based Nuclei Feature Alignment.

Mutual-Complementing Framework for Nuclei Detection and Segmentation in Pathology Image

Unsupervised Domain Adaptation with Adversarial Learning for Liver Tumors Detection in Multi-phase CT Images

Semi-Supervised Nuclei Detection in Histopathology Images via Location-Aware Adversarial Image Reconstruction

Nuclei segmentation using attention aware and adversarial networks

Dual Consistency Semi-Supervised Nuclei Detection Via Global Regularization and Local Adversarial Learning.

Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images

Reversed domain adaptation for nuclei segmentation-based pathological image classification

Learning to Generalize over Subpartitions for Heterogeneity-Aware Domain Adaptive Nuclei Segmentation

A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images

Position-based Anchor Optimization for Point Supervised Dense Nuclei Detection

Attention augmented distance regression and classification network for nuclei instance segmentation and type classification in histology images

Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images

Contrastive and uncertainty-aware nuclei segmentation and classification

Weakly Supervised Deep Nuclei Segmentation Using Partial Points Annotation in Histopathology Images

Nuclei segmentation of HE stained histopathological images based on feature global delivery connection network

Unsupervised domain adaptation for nuclei segmentation: Adapting from hematoxylin & eosin stained slides to immunohistochemistry stained slides using a curriculum approach

Revisiting Adaptive Cellular Recognition Under Domain Shifts: A Contextual Correspondence View