Abstract:Importance: Assessing endoscopic disease severity in ulcerative colitis (UC) is a key element in determining therapeutic response, but its use in clinical practice is limited by the requirement for experienced human reviewers.Objective: To determine whether deep learning models can grade the endoscopic severity of UC as well as experienced human reviewers.Design, Setting, and Participants: In this diagnostic study, retrospective grading of endoscopic images using the 4-level Mayo subscore was performed by 2 independent reviewers with score discrepancies adjudicated by a third reviewer. Using 16 514 images from 3082 patients with UC who underwent colonoscopy at a single tertiary care referral center in the United States between January 1, 2007, and December 31, 2017, a 159-layer convolutional neural network (CNN) was constructed as a deep learning model to train and categorize images into 2 clinically relevant groups: remission (Mayo subscore 0 or 1) and moderate to severe disease (Mayo subscore, 2 or 3). Ninety percent of the cohort was used to build the model and 10% was used to test it; the process was repeated 10 times. A set of 30 full-motion colonoscopy videos, unseen by the model, was then used for external validation to mimic real-world application.Main Outcomes and Measures: Model performance was assessed using area under the receiver operating curve (AUROC), sensitivity and specificity, positive predictive value (PPV), and negative predictive value (NPV). Kappa statistics (κ) were used to measure agreement of the CNN relative to adjudicated human reference cores.Results: The authors included 16 514 images from 3082 unique patients (median [IQR] age, 41.3 [26.1-61.8] years, 1678 [54.4%] female), with 3980 images (24.1%) classified as moderate-to-severe disease by the adjudicated reference score. The CNN was excellent for distinguishing endoscopic remission from moderate-to-severe disease with an AUROC of 0.966 (95% CI, 0.967-0.972); a PPV of 0.87 (95% CI, 0.85-0.88) with a sensitivity of 83.0% (95% CI, 80.8%-85.4%) and specificty of 96.0% (95% CI, 95.1%-97.1%); and NPV of 0.94 (95% CI, 0.93-0.95). Weighted κ agreement between the CNN and the adjudicated reference score was also good for identifying exact Mayo subscores (κ = 0.84; 95% CI, 0.83-0.86) and was similar to the agreement between experienced reviewers (κ = 0.86; 95% CI, 0.85-0.87). Applying the CNN to entire colonoscopy videos had similar accuracy for identifying moderate to severe disease (AUROC, 0.97; 95% CI, 0.963-0.969).Conclusions and Relevance: This study found that deep learning model performance was similar to experienced human reviewers in grading endoscopic severity of UC. Given its scalability, this approach could improve the use of colonoscopy for UC in both research and routine practice.

S841 Training Machine Learning Models for the Assessment of the Endoscopic Mayo Score in Ulcerative Colitis: A Systematic Review

P123 Development of an end-to-end Mayo endoscopic score machine learning algorithm for use in clinical trials of ulcerative colitis

Development and Validation of a Deep Learning-Based Approach to Predict the Mayo Endoscopic Score of Ulcerative Colitis

Performance of a Deep Learning Model vs Human Reviewers in Grading Endoscopic Disease Severity of Patients With Ulcerative Colitis

Predictive models for endoscopic disease activity in patients with ulcerative colitis: Practical machine learning-based modeling and interpretation

Improving the Computer-Aided Estimation of Ulcerative Colitis Severity According to Mayo Endoscopic Score by Using Regression-Based Deep Learning

Diagnostic accuracy of convolutional neural network-based machine learning algorithms in endoscopic severity prediction of ulcerative colitis: a systematic review and meta-analysis

P228 Machine learning models utilizing simple laboratory tests may assess endoscopic activity in ulcerative colitis

P595 Feasibility and performance of a fully automated endoscopic disease severity grading tool for ulcerative colitis using unaltered multisite videos

P487 Automated Scoring of Patient Endoscopy Videos Using Deep Learning Techniques: A Promising Approach for Clinical Trials in Inflammatory Bowel Disease

Accurate, Robust, and Scalable Machine Abstraction of Mayo Endoscopic Subscores From Colonoscopy Reports

P213 ArgesMES: A high-performance, generalizable AI model for scoring Ulcerative Colitis severity from endoscopy videos

Enhancing Calprotectin's Predictive Power as a Biomarker of Endoscopic Activity in Ulcerative Colitis: A Machine Learning Use Case

Automatic estimation of ulcerative colitis severity from endoscopy videos using ordinal multi-instance learning

A predictive machine-learning model for clinical decision-making in washed microbiota transplantation on ulcerative colitis

P598 Validating and Enhancing Real-time Disease Severity Classification in Ulcerative Colitis: Artificial Intelligence as a Second Opinion Trigger

A novel deep learning-based computer-aided diagnosis system for predicting inflammatory activity in ulcerative colitis

Inter- and Intraobserver Variability on Endoscopic Scoring Systems in Crohn’s Disease and Ulcerative Colitis: A Systematic Review and Meta-Analysis

MACHINE LEARNING-BASED PREDICTION OF GEBOES SCORE AND HISTOLOGIC IMPROVEMENT AND REMISSION THRESHOLDS IN ULCERATIVE COLITIS

Building an AI Support Tool for Real-time Ulcerative Colitis Diagnosis

Artificial intelligence and machine learning technologies in ulcerative colitis