Abstract:Importance: Assessing endoscopic disease severity in ulcerative colitis (UC) is a key element in determining therapeutic response, but its use in clinical practice is limited by the requirement for experienced human reviewers.Objective: To determine whether deep learning models can grade the endoscopic severity of UC as well as experienced human reviewers.Design, Setting, and Participants: In this diagnostic study, retrospective grading of endoscopic images using the 4-level Mayo subscore was performed by 2 independent reviewers with score discrepancies adjudicated by a third reviewer. Using 16 514 images from 3082 patients with UC who underwent colonoscopy at a single tertiary care referral center in the United States between January 1, 2007, and December 31, 2017, a 159-layer convolutional neural network (CNN) was constructed as a deep learning model to train and categorize images into 2 clinically relevant groups: remission (Mayo subscore 0 or 1) and moderate to severe disease (Mayo subscore, 2 or 3). Ninety percent of the cohort was used to build the model and 10% was used to test it; the process was repeated 10 times. A set of 30 full-motion colonoscopy videos, unseen by the model, was then used for external validation to mimic real-world application.Main Outcomes and Measures: Model performance was assessed using area under the receiver operating curve (AUROC), sensitivity and specificity, positive predictive value (PPV), and negative predictive value (NPV). Kappa statistics (κ) were used to measure agreement of the CNN relative to adjudicated human reference cores.Results: The authors included 16 514 images from 3082 unique patients (median [IQR] age, 41.3 [26.1-61.8] years, 1678 [54.4%] female), with 3980 images (24.1%) classified as moderate-to-severe disease by the adjudicated reference score. The CNN was excellent for distinguishing endoscopic remission from moderate-to-severe disease with an AUROC of 0.966 (95% CI, 0.967-0.972); a PPV of 0.87 (95% CI, 0.85-0.88) with a sensitivity of 83.0% (95% CI, 80.8%-85.4%) and specificty of 96.0% (95% CI, 95.1%-97.1%); and NPV of 0.94 (95% CI, 0.93-0.95). Weighted κ agreement between the CNN and the adjudicated reference score was also good for identifying exact Mayo subscores (κ = 0.84; 95% CI, 0.83-0.86) and was similar to the agreement between experienced reviewers (κ = 0.86; 95% CI, 0.85-0.87). Applying the CNN to entire colonoscopy videos had similar accuracy for identifying moderate to severe disease (AUROC, 0.97; 95% CI, 0.963-0.969).Conclusions and Relevance: This study found that deep learning model performance was similar to experienced human reviewers in grading endoscopic severity of UC. Given its scalability, this approach could improve the use of colonoscopy for UC in both research and routine practice.

Using deep learning and explainable artificial intelligence to assess the severity of gastroesophageal reflux disease according to the Los Angeles Classification System

Multi-Class Gastroesophageal Reflux Disease Classification System Using Deep Learning Techniques

An Improved Endoscopic Automatic Classification Model for Gastroesophageal Reflux Disease Using Deep Learning Integrated Machine Learning

Using deep learning to assess the function of gastroesophageal flap valve according to the Hill classification system

Real-Time Multi-Label Upper Gastrointestinal Anatomy Recognition from Gastroscope Videos

Explainable AI based automated segmentation and multi-stage classification of gastroesophageal reflux using machine learning techniques

Expert-level classification of gastritis by endoscopy using deep learning: a multicenter diagnostic trial

Deep Learning for Automatic Diagnosis of Gastric Dysplasia Using Whole-Slide Histopathology Images in Endoscopic Specimens

Deep Learning for Classification and Localization of Early Gastric Cancer in Endoscopic Images.

Explainable artificial intelligence incorporated with domain knowledge diagnosing early gastric neoplasms under white light endoscopy

Development and Validation of a Deep Learning-Based Approach to Predict the Mayo Endoscopic Score of Ulcerative Colitis

Deep Learning for Laryngopharyngeal Reflux Diagnosis

A review of deep learning methods for gastrointestinal diseases classification applied in computer-aided diagnosis system

Deep Learning and Gastric Cancer: Systematic Review of AI-Assisted Endoscopy

Explainability of a Deep Learning-Based Classification Model for ANCA-Associated Glomerulonephritis

Endoscopic Image Classification Based on Explainable Deep Learning

Identification of Upper Gastrointestinal Diseases during Screening Gastroscopy through Deep Convolutional Neural Network Algorithm

Automatic Classification of Gastric Lesions in Gastroscopic Images Using a Lightweight Deep Learning Model with Attention Mechanism and Cost-Sensitive Learning

Deep Learning-Based Anatomical Site Classification for Upper Gastrointestinal Endoscopy

Identification of Upper GI Diseases During Screening Gastroscopy Using a Deep Convolutional Neural Network Algorithm.

Performance of a Deep Learning Model vs Human Reviewers in Grading Endoscopic Disease Severity of Patients With Ulcerative Colitis