Abstract:Importance: Deep learning-based methods, such as the sliding window approach for cropped-image classification and heuristic aggregation for whole-slide inference, for analyzing histological patterns in high-resolution microscopy images have shown promising results. These approaches, however, require a laborious annotation process and are fragmented.Objective: To evaluate a novel deep learning method that uses tissue-level annotations for high-resolution histological image analysis for Barrett esophagus (BE) and esophageal adenocarcinoma detection.Design, Setting, and Participants: This diagnostic study collected deidentified high-resolution histological images (N = 379) for training a new model composed of a convolutional neural network and a grid-based attention network. Histological images of patients who underwent endoscopic esophagus and gastroesophageal junction mucosal biopsy between January 1, 2016, and December 31, 2018, at Dartmouth-Hitchcock Medical Center (Lebanon, New Hampshire) were collected.Main Outcomes and Measures: The model was evaluated on an independent testing set of 123 histological images with 4 classes: normal, BE-no-dysplasia, BE-with-dysplasia, and adenocarcinoma. Performance of this model was measured and compared with that of the current state-of-the-art sliding window approach using the following standard machine learning metrics: accuracy, recall, precision, and F1 score.Results: Of the independent testing set of 123 histological images, 30 (24.4%) were in the BE-no-dysplasia class, 14 (11.4%) in the BE-with-dysplasia class, 21 (17.1%) in the adenocarcinoma class, and 58 (47.2%) in the normal class. Classification accuracies of the proposed model were 0.85 (95% CI, 0.81-0.90) for the BE-no-dysplasia class, 0.89 (95% CI, 0.84-0.92) for the BE-with-dysplasia class, and 0.88 (95% CI, 0.84-0.92) for the adenocarcinoma class. The proposed model achieved a mean accuracy of 0.83 (95% CI, 0.80-0.86) and marginally outperformed the sliding window approach on the same testing set. The F1 scores of the attention-based model were at least 8% higher for each class compared with the sliding window approach: 0.68 (95% CI, 0.61-0.75) vs 0.61 (95% CI, 0.53-0.68) for the normal class, 0.72 (95% CI, 0.63-0.80) vs 0.58 (95% CI, 0.45-0.69) for the BE-no-dysplasia class, 0.30 (95% CI, 0.11-0.48) vs 0.22 (95% CI, 0.11-0.33) for the BE-with-dysplasia class, and 0.67 (95% CI, 0.54-0.77) vs 0.58 (95% CI, 0.44-0.70) for the adenocarcinoma class. However, this outperformance was not statistically significant.Conclusions and Relevance: Results of this study suggest that the proposed attention-based deep neural network framework for BE and esophageal adenocarcinoma detection is important because it is based solely on tissue-level annotations, unlike existing methods that are based on regions of interest. This new model is expected to open avenues for applying deep learning to digital pathology.

Attention-based whole-slide image compression achieves pathologist-level pre-screening of multi-organ routine histopathology biopsies

A promising deep learning-assistive algorithm for histopathological screening of colorectal cancer

Robust whole slide image analysis for cervical cancer screening using deep learning

Computer-aided Detection of Squamous Carcinoma of the Cervix in Whole Slide Images

Scalable deep learning artificial intelligence histopathology slide analysis and validation

Clinical-grade computational pathology using weakly supervised deep learning on whole slide images

Automated curation of large‐scale cancer histopathology image datasets using deep learning

Deep Learning Classification of Colorectal Lesions Based on Whole Slide Images

Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent

Accuracy and Efficiency of Deep-Learning–Based Automation of Dual Stain Cytology in Cervical Cancer Screening

Deep Learning-Based Sparse Whole-Slide Image Analysis for the Diagnosis of Gastric Intestinal Metaplasia

Automated Diagnosis of Lymphoma with Digital Pathology Images Using Deep Learning

Deep learning for multi-class semantic segmentation enables colorectal cancer detection and classification in digital pathology images

Attention-Based Deep Neural Networks for Detection of Cancerous and Precancerous Esophagus Tissue on Histopathological Slides

Multicenter automatic detection of invasive carcinoma on breast whole slide images

Weakly supervised annotation-free cancer detection and prediction of genotype in routine histopathology

Advancing automated digital pathology by rapid spectral imaging and AI for nuclear segmentation

Augmenting the Pathology Lab: An Intelligent Whole Slide Image Classification System for the Real World

A Generalized Deep Learning Framework for Whole-Slide Image Segmentation and Analysis

Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis

Data-efficient and weakly supervised computational pathology on whole-slide images