Abstract:Esophagogastroduodenoscopy (EGD) is a critical step in the diagnosis of upper gastrointestinal disorders. However, due to inexperience or high workload, there is a wide variation in EGD performance by endoscopists. Variations in performance may result in exams that do not completely cover all anatomical locations of the stomach, leading to a potential risk of missed diagnosis of gastric diseases. Numerous guidelines or expert consensus have been proposed to assess and optimize the quality of endoscopy. However, there is a lack of mature and robust methods to accurately apply to real clinical real-time video environments. In this paper, we innovatively define the problem of recognizing anatomical locations in videos as a multi-label recognition task. This can be more consistent with the model learning of image-to-label mapping relationships. We propose a combined structure of a deep learning model (GL-Net) that combines a graph convolutional network (GCN) with long short-term memory (LSTM) networks to both extract label features and correlate temporal dependencies for accurate real-time anatomical locations identification in gastroscopy videos. Our methodological evaluation dataset is based on complete videos of real clinical examinations. A total of 29,269 images from 49 videos were collected as a dataset for model training and validation. Another 1736 clinical videos were retrospectively analyzed and evaluated for the application of the proposed model. Our method achieves 97.1% mean accuracy (mAP), 95.5% mean per-class accuracy and 93.7% average overall accuracy in a multi-label classification task, and is able to process these videos in real-time at 29.9 FPS. In addition, based on our approach, we designed a system to monitor routine EGD videos in detail and perform statistical analysis of the operating habits of endoscopists, which can be a useful tool to improve the quality of clinical endoscopy.

Upper gastrointestinal anatomy detection with multi-task convolutional neural networks

Real-Time Multi-Label Upper Gastrointestinal Anatomy Recognition from Gastroscope Videos

Tu1075 DEEP CONVOLUTIONAL NEURAL NETWORKS FOR RECOGNITION OF ATROPHIC GASTRITIS AND INTESTINAL METAPLASIA BASED ON ENDOSCOPY IMAGES

Real-Time Gastric Polyp Detection Using Convolutional Neural Networks

Channel Separation-Based Network for the Automatic Anatomical Site Recognition Using Endoscopic Images.

Gastric Location Classification During Esophagogastroduodenoscopy Using Deep Neural Networks.

Automated Disease Detection in Gastroscopy Videos Using Convolutional Neural Networks

Multi-Stream Scale-Insensitive Convolutional And Recurrent Neural Networks For Liver Tumor Detection In Dynamic Ct Images

Diagnosis of routine endoscopic images of gastric lesions through a deep convolutional neural network

Content‐based Gastric Image Retrieval Using Convolutional Neural Networks

Identification of Upper GI Diseases During Screening Gastroscopy Using a Deep Convolutional Neural Network Algorithm.

Identification of Upper Gastrointestinal Diseases during Screening Gastroscopy through Deep Convolutional Neural Network Algorithm

A Deep Neural Network Improves Endoscopic Detection of Early Gastric Cancer Without Blind Spots

Deep learning-based anatomical position recognition for gastroscopic examination

Large-scale Gastric Cancer Screening and Localization Using Multi-task Deep Neural Network

Diagnosis of Gastric Lesions Through a Deep Convolutional Neural Network.

Automatic Classification of Esophageal Lesions in Endoscopic Images Using a Convolutional Neural Network

Artificial intelligence based on deep learning for automatic detection of early gastric cancer

Deep Learning-Based Anatomical Site Classification for Upper Gastrointestinal Endoscopy

A Hierarchical Multi-Task Approach to Gastrointestinal Image Analysis

Tu2005 DEVELOPMENT AND VALIDATION OF A DEEP LEARNING ALGORITHM FOR DETECTION AND RECOGNITION OF PRECANCEROUS LESION IN ESOPHAGUS