Abstract:Background: Light microscopy is a widely used technique in cell biology due to its satisfactory resolution for cellular structure analysis, prevalent availability of fluorescent probes for staining, and compatibility for the dynamic analysis of live cells. However, the segmentation of cells and nuclei from microscopic images is not a straightforward process because it has several challenges such as high variation in morphology and shape, the presence of noise and diverse contrast in backgrounds, clustering or overlapping nature of cells. Dealing with these challenges and facilitating more reliable analysis necessitates the implementation of computer-aided methods that leverage image processing techniques and deep learning algorithms. The major goal of this study is to propose a model, for instance segmentation of cells and nuclei, applying the most cutting-edge deep learning techniques. Methods: A fine-tuned You Only Look at Once version 9 extended (YOLOv9-E) model is initially applied as a prompt generator to generate bounding box prompts. Using the generated prompts, a pre-trained segment anything model (SAM) is subsequently applied through zero-short inferencing to produce raw segmentation masks. These segmentation masks are then refined using non-max suppression and simple image processing methods such as image addition and morphological processing. The proposed method is developed and evaluated using an open-sourced dataset called Expert Visual Cell Annotation (EVICAN), which is relatively large and contains 4,738 microscopy images extracted from cross organs using different protocols. Results: Based on the evaluation results on three different levels of EVICAN test sets, the proposed method demonstrates noticeable performances showing average mAP50 [mean average precision at intersection over union (IoU) =0.50] scores of 96.25, 95.05, and 94.18 for cell segmentation, and 68.04, 54.66, and 38.29 for nucleus segmentation on easy, medium, and difficult test sets, respectively. Conclusions: Our proposed method for instance segmentation of cells and nuclei provided favorable performance compared to the existing methods in literature, indicating its potential utility as an assistive tool for cell culture experts, facilitating prompt and reliable analysis.

Cyto R-CNN and CytoNuke Dataset: Towards reliable whole-cell segmentation in bright-field histological images

Unsupervised high-throughput segmentation of cells and cell nuclei in quantitative phase images

aiSEGcell: User-friendly deep learning-based segmentation of nuclei in transmitted light images

Cervical Nuclei Segmentation in Whole Slide Histopathology Images Using Convolution Neural Network

Segmentation of cervical cell nuclei in high-resolution microscopic images: A new algorithm and a web-based software framework

Evaluation Of Deep Learning Strategies For Nucleus Segmentation In Fluorescence Images

CellSeg: a robust, pre-trained nucleus segmentation and pixel quantification software for highly multiplexed fluorescence images

Real-time and accurate deep learning-based multi-organ nucleus segmentation in histology images

Deep Learning Model for Cell Nuclei Segmentation and Lymphocyte Identification in Whole Slide Histology Images

CNSeg: A dataset for cervical nuclear segmentation

Nuclei R-CNN: Improve Mask R-CNN for Nuclei Segmentation

A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images

Dataset of Segmented Nuclei in Hematoxylin and Eosin Stained Histopathology Images of 10 Cancer Types

Instance segmentation of cells and nuclei from multi-organ cross-protocol microscopic images

NISNet3D: three-dimensional nuclear synthesis and instance segmentation for fluorescence microscopy images

CryoNuSeg: A dataset for nuclei instance segmentation of cryosectioned H&E-stained histological images

CISCA and CytoDArk0: a Cell Instance Segmentation and Classification method for histo(patho)logical image Analyses and a new, open, Nissl-stained dataset for brain cytoarchitecture studies

Cell Nuclei Segmentation in Cryonuseg dataset using Nested Unet with EfficientNet Encoder

FRE-Net: Full-region enhanced network for nuclei segmentation in histopathology images

Efficient automatic 3D segmentation of cell nuclei for high-content screening

Nucleus segmentation: towards automated solutions