Abstract:Endoscopy is a routine imaging technique used for both diagnosis and minimally invasive surgical treatment. Artifacts such as motion blur, bubbles, specular reflections, floating objects and pixel saturation impede the visual interpretation and the automated analysis of endoscopy videos. Given the widespread use of endoscopy in different clinical applications, robust and reliable identification of such artifacts and the automated restoration of corrupted video frames is a fundamental medical imaging problem. Existing state-of-the-art methods only deal with the detection and restoration of selected artifacts. However, typically endoscopy videos contain numerous artifacts which motivates to establish a comprehensive solution.In this paper, a fully automatic framework is proposed that can: 1) detect and classify six different artifacts, 2) segment artifact instances that have indefinable shapes, 3) provide a quality score for each frame, and 4) restore partially corrupted frames. To detect and classify different artifacts, the proposed framework exploits fast, multi-scale and single stage convolution neural network detector. In addition, we use an encoder-decoder model for pixel-wise segmentation of irregular shaped artifacts. A quality score is introduced to assess video frame quality and to predict image restoration success. Generative adversarial networks with carefully chosen regularization and training strategies for discriminator-generator networks are finally used to restore corrupted frames.The detector yields the highest mean average precision (mAP) of 45.7 and 34.7, respectively for 25% and 50% IoU thresholds, and the lowest computational time of 88 ms allowing for near real-time processing. The restoration models for blind deblurring, saturation correction and inpainting demonstrate significant improvements over previous methods. On a set of 10 test videos, an average of 68.7% of video frames successfully passed the quality score (<math>≥0.9</math>) after applying the proposed restoration framework thereby retaining 25% more frames compared to the raw videos. The importance of artifacts detection and their restoration on improved robustness of image analysis methods is also demonstrated in this work.

Automatic gastroscopy video quality assessment

Real-Time Multi-Label Upper Gastrointestinal Anatomy Recognition from Gastroscope Videos

Human Visual System-Based Fundus Image Quality Assessment of Portable Fundus Camera Photographs.

Subjective and Objective Quality Assessment of Gastrointestinal Endoscopy Images: from Manual Operation to Artificial Intelligence

Automated Disease Detection in Gastroscopy Videos Using Convolutional Neural Networks

An Automated Video Analysis System for Retrospective Assessment and Real-Time Monitoring of Endoscopic Procedures (with Video)

A deep learning framework for quality assessment and restoration in video endoscopy

Efficient video indexing for monitoring disease activity and progression in the upper gastrointestinal tract

Automatic Image Unfolding and Stitching Framework for Esophageal Lining Video Based on Density-Weighted Feature Matching

Computer-aided Diagnosis Based on Enhancement of Degraded Fundus Photographs.

A comprehensive analysis of classification methods in gastrointestinal endoscopy imaging

Global and Local Panoramic Views for Gastroscopy: an Assisted Method of Gastroscopic Lesion Surveillance.

Deep sparse feature selection for computer aided endoscopy diagnosis

Detecting and Locating Gastrointestinal Anomalies Using Deep Learning and Iterative Cluster Unification

Self-supervised monocular depth estimation for gastrointestinal endoscopy

Evaluation Tool to Diagnose Faults and Discrepancy in Semi-Automated or Manual Annotations in Ultrasound Cine Loops (Videos)

Subjective and Objective Quality Assessment of Colonoscopy Videos

Automated Scene Classification in Endoscopy Videos Using Convolutional Neural Networks

A comprehensive study on automatic non‐informative frame detection in colonoscopy videos

A multi-label dataset and its evaluation for automated scoring system for cleanliness assessment in video capsule endoscopy

Automatic Detection of Early Gastrointestinal Cancer Lesions Based on Optimal Feature Extraction from Gastroscopic Images