CUPre: Cross-domain Unsupervised Pre-training for Few-Shot Cell Segmentation

Weibin Liao,Xuhong Li,Qingzhong Wang,Yanwu Xu,Zhaozheng Yin,Haoyi Xiong

2023-10-06

Abstract:While pre-training on object detection tasks, such as Common Objects in Contexts (COCO) [1], could significantly boost the performance of cell segmentation, it still consumes on massive fine-annotated cell images [2] with bounding boxes, masks, and cell types for every cell in every image, to fine-tune the pre-trained model. To lower the cost of annotation, this work considers the problem of pre-training DNN models for few-shot cell segmentation, where massive unlabeled cell images are available but only a small proportion is annotated. Hereby, we propose Cross-domain Unsupervised Pre-training, namely CUPre, transferring the capability of object detection and instance segmentation for common visual objects (learned from COCO) to the visual domain of cells using unlabeled images. Given a standard COCO pre-trained network with backbone, neck, and head modules, CUPre adopts an alternate multi-task pre-training (AMT2) procedure with two sub-tasks -- in every iteration of pre-training, AMT2 first trains the backbone with cell images from multiple cell datasets via unsupervised momentum contrastive learning (MoCo) [3], and then trains the whole model with vanilla COCO datasets via instance segmentation. After pre-training, CUPre fine-tunes the whole model on the cell segmentation task using a few annotated images. We carry out extensive experiments to evaluate CUPre using LIVECell [2] and BBBC038 [4] datasets in few-shot instance segmentation settings. The experiment shows that CUPre can outperform existing pre-training methods, achieving the highest average precision (AP) for few-shot cell segmentation and detection.

Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The paper aims to address the high cost of fine-grained annotation in cell segmentation and proposes a new method called CUPre (Cross-domain Unsupervised Pre-training). Specifically, the paper focuses on the following aspects: 1. **Reducing Annotation Cost**: Traditional methods require a large number of finely annotated cell images to train models, while CUPre attempts to reduce this cost by utilizing unlabeled cell images. 2. **Cross-domain Adaptation**: Due to the significant differences between natural images (such as those in the COCO dataset) and cell images under a microscope, the paper proposes a cross-domain pre-training method to mitigate the impact of these differences. 3. **Few-shot Learning**: CUPre can effectively perform cell segmentation tasks with only a small number of annotated images. The main contributions of the paper include: - Proposing a new cross-domain unsupervised pre-training framework, CUPre, for few-shot cell segmentation tasks. - Designing an Alternating Multi-task Pre-training (AMT2) process that combines the advantages of self-supervised learning and supervised learning. - Experimental results show that CUPre achieves significantly better performance than existing methods on multiple benchmark datasets (such as LIVECell and BBBC038), especially in few-shot learning settings. For example, when using only 5% of the annotated images in the LIVECell dataset, CUPre achieved 41.5% APbbox, while the COCO pre-trained model and the MoCo-based LIVECell pre-trained model only reached 40.0% and 8.3%, respectively. In summary, the paper is primarily dedicated to developing a method that can efficiently perform cell segmentation with less annotated data and validates its superior performance through experiments.

CUPre: Cross-domain Unsupervised Pre-training for Few-Shot Cell Segmentation

Joint Feature Learning for Cell Segmentation Based on Multi-scale Convolutional U-Net.

Scellseg: a Style-Aware Cell Instance Segmentation Tool with Pre-Training and Contrastive Fine-Tuning

Pixel Matching Network for Cross-Domain Few-Shot Segmentation

Adept: Annotation-denoising Auxiliary Tasks with Discrete Cosine Transform Map and Keypoint for Human-Centric Pretraining

Cell segmentation without annotation by unsupervised domain adaptation based on cooperative self-learning

Unsupervised Pre-training with Language-Vision Prompts for Low-Data Instance Segmentation

CellTranspose: Few-shot Domain Adaptation for Cellular Instance Segmentation

Auto-CSC: A Transfer Learning Based Automatic Cell Segmentation and Count Framework

Res2-UNeXt: a novel deep learning framework for few-shot cell image segmentation

UCC: Uncertainty Guided Cross-head Cotraining for Semi-Supervised Semantic Segmentation.

Learn to segment single cells with deep distance estimator and deep cell detector

CRNet: Cross-Reference Networks for Few-Shot Segmentation

Cell segmentation for immunofluorescence multiplexed images using two-stage domain adaptation and weakly labeled data for pre-training

Background Clustering Pre-training for Few-shot Segmentation

DenseMP: Unsupervised Dense Pre-training for Few-shot Medical Image Segmentation.

Multi-dataset Pretraining: A Unified Model for Semantic Segmentation

Unsupervised Universal Image Segmentation

Multi-Path U-Net Architecture for Cell and Colony-Forming Unit Image Segmentation

Few-shot Segmentation with Duplex Network and Attention Augmented Module.

Cross attention redistribution with contrastive learning for few shot object detection