Abstract:Accurate nuclei segmentation is an essential foundation for various applications in computational pathology, including cancer diagnosis and treatment planning. Even slight variations in nuclei representations can significantly impact these downstream tasks. However, achieving accurate segmentation remains challenging due to factors like clustered nuclei, high intra-class variability in size and shape, resemblance to other cells, and color or contrast variations between nuclei and background. Despite the extensive utilization of Convolutional Neural Networks (CNNs) in medical image segmentation, they may have trouble capturing long-range dependencies crucial for accurate nuclei delineation. Transformers address this limitation but might miss essential low-level features. To overcome these limitations, we utilized CNN-Transformer-based techniques for nuclei segmentation in H&E stained histology images. In this work, we proposed two CNN-Transformer architectures, Nuclei Hybrid Vision Transformer (NucleiHVT) and Channel Boosted Nuclei Hybrid Vision Transformer (CB-NucleiHVT), that leverage the strengths of both CNNs and Transformers to effectively learn nuclei boundaries in multi-organ histology images. The first architecture, NucleiHVT is inspired by the UNet architecture and incorporates the dual attention mechanism to capture both multi-level and multi-scale context effectively. The CB-NucleiHVT network, on the other hand, utilizes the concept of channel boosting to learn diverse feature spaces, enhancing the model's ability to distinguish subtle variations in nuclei characteristics. Detailed evaluation of two medical image segmentation datasets shows that the proposed architectures outperform existing CNN-based, Transformer-based, and hybrid methods. The proposed networks demonstrated effective results both in terms of quantitative metrics, and qualitative visual assessment.

An Improved Hover-net for Nuclear Segmentation and Classification in Histopathology Images.

Mutual-Complementing Framework for Nuclei Detection and Segmentation in Pathology Image

FRE-Net: Full-region enhanced network for nuclei segmentation in histopathology images

HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images

Attentional Dilated Convolution Neural Network for Nuclei Segmentation in Histopathology Images

Nuclei segmentation of HE stained histopathological images based on feature global delivery connection network

High-resolution deep transferred ASPPU-Net for nuclei segmentation of histopathology images

A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images

Attention augmented distance regression and classification network for nuclei instance segmentation and type classification in histology images

A Dual-Path Instance Segmentation Network Based on Nuclei Contour in Histology Image

Channel Boosted CNN-Transformer-based Multi-Level and Multi-Scale Nuclei Segmentation

A Hybrid-Attention Nested UNet for Nuclear Segmentation in Histopathological Images

HoverFast: an accurate, high-throughput, clinically deployable nuclear segmentation tool for brightfield digital pathology images

MDC-net: A new convolutional neural network for nucleus segmentation in histopathology images with distance maps and contour information

Real-time and accurate deep learning-based multi-organ nucleus segmentation in histology images

A Robust Deep Learning Approach for Joint Nuclei Detection and Cell Classification in Pan-Cancer Histology Images

AugHover-Net: Augmenting Hover-net for Nucleus Segmentation and Classification

Cervical Nuclei Segmentation in Whole Slide Histopathology Images Using Convolution Neural Network

MATNet: a multi-attention transformer network for nuclei segmentation in thymoma histopathology images

Nuclei Segmentation in Histopathology Images Using Rotation Equivariant and Multi-level Feature Aggregation Neural Network.

SFE-Net: Spatial-frequency Enhancement Network for Robust Nuclei Segmentation in Histopathology Images