Abstract:Esophagogastroduodenoscopy (EGD) is a critical step in the diagnosis of upper gastrointestinal disorders. However, due to inexperience or high workload, there is a wide variation in EGD performance by endoscopists. Variations in performance may result in exams that do not completely cover all anatomical locations of the stomach, leading to a potential risk of missed diagnosis of gastric diseases. Numerous guidelines or expert consensus have been proposed to assess and optimize the quality of endoscopy. However, there is a lack of mature and robust methods to accurately apply to real clinical real-time video environments. In this paper, we innovatively define the problem of recognizing anatomical locations in videos as a multi-label recognition task. This can be more consistent with the model learning of image-to-label mapping relationships. We propose a combined structure of a deep learning model (GL-Net) that combines a graph convolutional network (GCN) with long short-term memory (LSTM) networks to both extract label features and correlate temporal dependencies for accurate real-time anatomical locations identification in gastroscopy videos. Our methodological evaluation dataset is based on complete videos of real clinical examinations. A total of 29,269 images from 49 videos were collected as a dataset for model training and validation. Another 1736 clinical videos were retrospectively analyzed and evaluated for the application of the proposed model. Our method achieves 97.1% mean accuracy (mAP), 95.5% mean per-class accuracy and 93.7% average overall accuracy in a multi-label classification task, and is able to process these videos in real-time at 29.9 FPS. In addition, based on our approach, we designed a system to monitor routine EGD videos in detail and perform statistical analysis of the operating habits of endoscopists, which can be a useful tool to improve the quality of clinical endoscopy.

Abnormal Region Detection in Gastroscopic Images by Combining Classifiers on Neighboring Patches

Real-Time Multi-Label Upper Gastrointestinal Anatomy Recognition from Gastroscope Videos

Gastroscopic Image Graph: Application to Noninvasive Multitarget Tracking under Gastroscopy

Gastrointestinal image stitching based on improved unsupervised algorithm

A Local Region Proposals Approach to Instance Segmentation for Intestinal Polyp Detection

Homographic Patch Feature Transform: A Robustness Registration for Gastroscopic Surgery.

Automated Identification of Human Gastrointestinal Tract Abnormalities Based on Deep Convolutional Neural Network with Endoscopic Images

Lesion Detection of Electronic Gastroscope Images Based on Multiscale Texture Feature

Integrating Deep Feature Extraction and Hybrid ResNet-DenseNet Model for Multi-Class Abnormality Detection in Endoscopic Images

Automated Disease Detection in Gastroscopy Videos Using Convolutional Neural Networks

Lesion Detection of Gastroscopic Images Based on Cost-Sensitive Boosting.

Gastrointestinal Stromal Tumors Diagnosis on Multi-Center Endoscopic Ultrasound Images Using Multi-Scale Image Normalization and Transfer Learning

Automated gastrointestinal abnormalities detection from endoscopic images

Early gastric cancer detection and lesion segmentation based on deep learning and gastroscopic images

Application of Improved Mask R-CNN Algorithm Based on Gastroscopic Image in Detection of Early Gastric Cancer

FusionNet: Incorporating Shape and Texture for Abnormality Detection in 3D Abdominal CT Scans

Enhancing Diagnostic Precision in Gastric Bleeding through Automated Lesion Segmentation: A Deep DuS-KFCM Approach

PatchCL-AE: Anomaly detection for medical images using patch-wise contrastive learning-based auto-encoder

A Comparative Analysis of Optimization Algorithms for Gastrointestinal Abnormalities Recognition and Classification Based on Ensemble XcepNet23 and ResNet18 Features

Accurate Pathological Patch Classification with Regression

Abnormality Detection in Mammography using Deep Convolutional Neural Networks