Multi-Stain Multi-Level Convolutional Network for Multi-Tissue Breast Cancer Image Segmentation

Akash Modi,Sumit Kumar Jha,Purnendu Mishra,Rajiv Kumar,Kiran Aatre,Gursewak Singh,Shubham Mathur
2024-06-09
Abstract:Digital pathology and microscopy image analysis are widely employed in the segmentation of digitally scanned IHC slides, primarily to identify cancer and pinpoint regions of interest (ROI) indicative of tumor presence. However, current ROI segmentation models are either stain-specific or suffer from the issues of stain and scanner variance due to different staining protocols or modalities across multiple labs. Also, tissues like Ductal Carcinoma in Situ (DCIS), acini, etc. are often classified as Tumors due to their structural similarities and color compositions. In this paper, we proposed a novel convolutional neural network (CNN) based Multi-class Tissue Segmentation model for histopathology whole-slide Breast slides which classify tumors and segments other tissue regions such as Ducts, acini, DCIS, Squamous epithelium, Blood Vessels, Necrosis, etc. as a separate class. Our unique pixel-aligned non-linear merge across spatial resolutions empowers models with both local and global fields of view for accurate detection of various classes. Our proposed model is also able to separate bad regions such as folds, artifacts, blurry regions, bubbles, etc. from tissue regions using multi-level context from different resolutions of WSI. Multi-phase iterative training with context-aware augmentation and increasing noise was used to efficiently train a multi-stain generic model with partial and noisy annotations from 513 slides. Our training pipeline used 12 million patches generated using context-aware augmentations which made our model stain and scanner invariant across data sources. To extrapolate stain and scanner invariance, our model was evaluated on 23000 patches which were for a completely new stain (Hematoxylin and Eosin) from a completely new scanner (Motic) from a different lab. The mean IOU was 0.72 which is on par with model performance on other data sources and scanners.
Computer Vision and Pattern Recognition,Artificial Intelligence,Image and Video Processing
What problem does this paper attempt to address?
This paper proposes a new method for addressing the problem of multi-organ breast cancer image segmentation. Existing models for breast cancer region of interest (ROI) segmentation are either specific to certain stains or affected by variations in staining and scanning devices across different laboratories. Moreover, due to similarities in structure and color, certain tissues such as ductal carcinoma in situ (DCIS) are often misclassified as tumors. In this paper, the authors propose a convolutional neural network (CNN) based multi-class tissue segmentation model for breast whole slide pathology images, which can separate tumors from other tissues such as ducts, glands, DCIS, squamous epithelium, vessels, necrosis, etc. The model utilizes pixel-aligned non-linear fusion and combines local and global views at different spatial resolutions to accurately detect various tissue classes. Through multi-stage iterative training and context-aware enhancement, the model can be effectively trained on partially annotated and noisy data, and it exhibits invariance to staining and scanning device variations from different sources. Evaluation on new stains and scanning devices demonstrates the model's generalization capability, with an average IoU of 0.72, proving its accuracy and wide application potential in assisting pathologists in evaluating breast cancer.