Abstract:Background and objective: Histologic assessment of the immune infiltrate in H&E slides is vital in diagnosing and managing inflammatory bowel diseases, but these assessments are subjective and time-consuming even for those with expertise. The development of deep learning models to aid in these assessments has been limited by the paucity of image data with reliably annotated immune cells available for training. Methods: To address these challenges, we developed a pipeline that automates the neutrophil and lymphocyte labeling in ROIs from digital H&E slides. The data included ROIs extracted from 19 digitized H&E slides and the same slides restained with immunohistochemistry. Our pipeline first delineates each nucleus in H&E ROIs. Using the colorimetric features of the immunohistochemical stains (red: neutrophils, green: lymphocytes) in the immunohistochemistry ROIs, each cell was labeled as a neutrophil, a lymphocyte, or another cell. The labels were then transferred to the corresponding H&E ROIs by image registration, and the ROI registration accuracy was assessed by the median target registration error resulting in a labeled dataset. The newly formed dataset (NeuLy-IHC) comprising 519 ROIs with 235,256 labeled cells (74,339 lymphocytes, 16,326 neutrophils and 144,591 other cells) was used to train the HoVer-Net(NeuLy) model. The performance of HoVer-Net(NeuLy) measured by DICE coefficient (segmentation accuracy) and F1-scores (classification accuracy), was compared to those achieved by HoVer-Net(MoNuSAC) and SMILE(MoNuSAC) publicly available models trained on cancer-containing ROIs from the MoNuSAC dataset with manual cell labeling and pathologists' annotations. Results: The 1.0 μm median target registration error of ROIs observed was low demonstrating robust transferring of cellular labels from immunohistochemistry ROIs to H&E ROIs. In the test set comprising 76 NeuLy-IHC and 78 MoNuSAC ROIs, the HoVer-Net(NeuLy) achieved a DICE coefficient of 0.861 and F1-sores of 0.827, 0.838, and 0.828, for neutrophils, lymphocytes, and other cells, respectively, outperforming the HoVer-Net(MoNuSAC)'s and SMILE(MoNuSAC)'s DICE coefficient and F1 scores for each cell category. Conclusions: We attribute the improved performance of HoVer-Net(NeuLy) to the larger number of immune cells in the NeuLy-IHC dataset (in total 5x more, including 21x more neutrophils) than in the MoNuSAC dataset. Despite being trained on data from inflammatory bowel disease specimens, our model maintained robust performance when tested on previously unseen data derived from cancer specimens. The NeuLy-IHC set provides opportunities for training accurate models to quantify the inflammatory infiltrate in digital histologic slides.

Nucleus subtype classification using inter-modality learning

Data-driven Nucleus Subclassification on Colon H&E using Style-transferred Digital Pathology

Data-driven Nucleus Subclassification on Colon Hematoxylin and Eosin Using Style-Transferred Digital Pathology

Simultaneous Semantic and Instance Segmentation for Colon Nuclei Identification and Counting

Cellular Segmentation and Composition in Routine Histology Images using Deep Learning

A Standardized Pipeline for Colon Nuclei Identification and Counting Challenge

UniCell: Universal Cell Nucleus Classification via Prompt Learning

AugHover-Net: Augmenting Hover-net for Nucleus Segmentation and Classification

Nuclear Segmentation and Classification Model with Imbalanced Classes for CoNiC Challenge

Separable-HoverNet and Instance-YOLO for Colon Nuclei Identification and Counting

CoNIC Challenge: Pushing the frontiers of nuclear detection, segmentation, classification and counting

Knowledge-Based Representation Learning for Nucleus Instance Classification From Histopathological Images

Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images

A deep learning method that identifies cellular heterogeneity using nanoscale nuclear features

A Robust Deep Learning Approach for Joint Nuclei Detection and Cell Classification in Pan-Cancer Histology Images

Nuclei panoptic segmentation and composition regression with multi-task deep neural networks

Hoechst Is All You Need: Lymphocyte Classification with Deep Learning

Immunohistochemistry annotations enhance AI identification of lymphocytes and neutrophils in digitized H&E slides from inflammatory bowel disease

Deep Learning Model for Cell Nuclei Segmentation and Lymphocyte Identification in Whole Slide Histology Images

Instance segmentation of cells and nuclei from multi-organ cross-protocol microscopic images

Staged Detection–Identification Framework for Cell Nuclei in Histopathology Images