Abstract:Background Endoscopy is increasingly important for the evaluation of patients with ulcerative colitis (UC). However, there were considerable differences in endoscopic assessment because of the endoscopists’ training experience, not only that, existing endoscopy scoring methods cannot reflect the inflammation’s details of the full-length endoscopic video; therefore, we sought to develop an automatic scoring system using deep learning technology for consistent and objective of endoscopic images and videos from patients with UC. Methods ResNet50 pre-trained on ImageNet was selected as our CNN main framework. We collected 5,875 endoscopic images and 16 full-length videos from patients with UC who underwent colonoscopy from January 2017 through March 2021as the dataset. The scoring system was trained by these images and could predict the Mayo score and UCEIS score by the feature it has learned on the training set. In addition, by using the predicted outcome, the CAD could show a ribbon in the shape of the colon which can reflect the conditions of the inflammation (location, severity). The scoring system is composed of clarity cleaning, similarity judgement and score prediction networks. This corresponding relation which combines the exiting endoscopy scores prediction, helps the system to output a new score with weighted calculation, as well as the lesions’ location and severity ratio. Results Two clinical reviewed our consistency statistics. The primary evaluation metric we used is quadratic weighted kappa (QWK). The trained CNN got high levels with 90.2% accuracy in the Mayo-scored task, while the QWK coefficient was 0.906 (95%CI, 0.880-0.933). The metrics of UCEIS-scored task were inspiring of which accuracy were 90.7%, 84.6%, 77.7%, QWK coefficient were 0.887 (95%CI, 0.860-0.915), 0.876 (95%CI, 0.844-0.907), 0.816 (95%CI, 0.739-0.892) for vascular pattern, erosions-and- ulcers and bleeding. Meanwhile, the CNN identified 16 full-length endoscopy videos and evaluated 1,360 local information from 100 intestine segments. The results of these localized intestinal tissue marked by the scored system were distributed in four ranges, and the quantities were 772(0.0-0.4), 124(0.72-2.35), 206(3.2-4.0), 258(5.2-8.4). It performed desirable compared with the clinical diagnosis. Conclusions The results of objective evaluations of UC based on this scoring system appeared acceptable. Compared with the previous scored methods (only refer to the most severe lesion and ignored other mild parts), this work is of clinical significance and will help clinicians by supplying a good capacity in identifying endoscopy data and stimulating the colon’s inflammatory environment.

Enteroscopy Crohn's disease auxiliary diagnosis system and method based on deep learning

Construction of Artificial Intelligence Assisted Diagnosis Model for Colonoscopy

Identification of ulcerative colitis and Crohn's disease based on spatial and bilinear attention network

Development and Validation of a Deep Neural Network for Accurate Identification of Endoscopic Images From Patients With Ulcerative Colitis and Crohn's Disease

Development and Validation of a Deep Learning-Based Approach to Predict the Mayo Endoscopic Score of Ulcerative Colitis

Identification of Upper Gastrointestinal Diseases during Screening Gastroscopy through Deep Convolutional Neural Network Algorithm

Tu1075 DEEP CONVOLUTIONAL NEURAL NETWORKS FOR RECOGNITION OF ATROPHIC GASTRITIS AND INTESTINAL METAPLASIA BASED ON ENDOSCOPY IMAGES

Identification of Upper GI Diseases During Screening Gastroscopy Using a Deep Convolutional Neural Network Algorithm.

IDDF2021-ABS-0206 A New Computer-Assisted Diagnosis Scoring System Based on Deep Learning for Predicting Inflammatory Activity from Patients with Ulcerative Colitis

DCDS-Net: Deep Transfer Network Based on Depth-Wise Separable Convolution with Residual Connection for Diagnosing Gastrointestinal Diseases

Design and implementation of deep learning algorithm for image discrimination of light field confocal endoscopy for esophageal cancer and gastric cancer

Development of a Convolutional Neural Network-Based Colonoscopy Image Assessment Model for Differentiating Crohn's Disease and Ulcerative Colitis

Tu2005 DEVELOPMENT AND VALIDATION OF A DEEP LEARNING ALGORITHM FOR DETECTION AND RECOGNITION OF PRECANCEROUS LESION IN ESOPHAGUS

Computer-aided diagnostic system based on deep learning for classifying colposcopy images

Improving the Computer-Aided Estimation of Ulcerative Colitis Severity According to Mayo Endoscopic Score by Using Regression-Based Deep Learning

Deep convolutional neural networks with ensemble learning and transfer learning for automated detection of gastrointestinal diseases

Automated Identification of Human Gastrointestinal Tract Abnormalities Based on Deep Convolutional Neural Network with Endoscopic Images

Detection of multiple lesions of gastrointestinal tract for endoscopy using artificial intelligence model: a pilot study

The Application of Deep Learning Based Diagnostic System to Cervical Squamous Intraepithelial Lesions Recognition in Colposcopy Images.

Multi-expert annotation of Crohn’s disease images of the small bowel for automatic detection using a convolutional recurrent attention neural network

Prediction of Crohn's Disease Based on Deep Feature Recognition