Abstract:Background and objective: Functional gastrointestinal disorders (FGIDs) are reported as worldwide gastrointestinal (GI) diseases. GI motility assessment can assist the diagnosis of patients with intestine motility dysfunction. Wireless capsule endoscopy (WCE) can acquire images in the gastrointestinal (GI) tract including the small intestine where other conventional endoscopes cannot penetrate, and WCE images can reveal GI motility. To generally analyze WCE frames, the high-precision registration of consecutive WCE frames is an absolute necessity. It is difficult and meaningless to register entire WCE frames on a pixel level due to the unpredictable and massive non-rigid deformation between consecutive frames, the low quality of imaging and the complex intestinal environment. Thus, the registration of region of interest (ROI) functioning in a feature level has more significance than entire frame registration. Methods: In this paper we present Timecylce-WCE, an end-to-end automatic registration approach of ROIs on WCE images. The clinicians can determine a ROI by drawing a bounding box in any WCE frame to be registered. This proposed approach is based on a deep-learning model of time-consistency in recurrent-registering, skip-registering and self-registering cycle, and it is fully unsupervised without any label. We incorporate the global correlation map with the local correlation map in matching the features, and a novel overall loss function is designed to enable the convergence of the model. As the output, a thin-plate spline (TPS) transformed region in the template frame is highly aligned with the query ROI in a finer-grained level. To the best of our knowledge this is the first time that a deep-learning-based registration method is proposed for WCE imaging motion. Results: To highlight the effectiveness of the proposed approach, our proposed method is compared with the existing non-deep-learning methods and tested in a validation dataset with labeled matching points. The presented method resulted in the best PCK@10 (Percentage of Correct Key-points, i.e., the predicted and the true joint is within the threshold - 10 pixels) of 66.49%. We also demonstrate that variants of design improved registration accuracy. Conclusions: From the experimental analysis, it is clear that our proposed method outperforms the other existing methods. This lays the groundwork for subsequent studies, such as GI motility assessment, and WCE image synthesis.

Second glance framework (secG): Enhanced ulcer detection with deep learning on a large wireless capsule endoscopy dataset

A Systematic Evaluation and Optimization of Automatic Detection of Ulcers in Wireless Capsule Endoscopy on a Large Dataset Using Deep Convolutional Neural Networks.

Deep Convolutional Neural Network for Ulcer Recognition in Wireless Capsule Endoscopy: Experimental Feasibility and Optimization

Automated Detection of Small Bowel Lesions Based on Capsule Endoscopy Using Deep Learning Algorithm

Color-Transfer-Enhanced Data Construction and Validation for Deep Learning-Based Upper Gastrointestinal Landmark Classification in Wireless Capsule Endoscopy

Capsule Endoscopy Image Classification with Deep Convolutional Neural Networks

Infinite Curriculum Learning for Efficiently Detecting Gastric Ulcers in WCE Images

LLCaps: Learning to Illuminate Low-Light Capsule Endoscopy with Curved Wavelet Attention and Reverse Diffusion

Efficiency Meets Accuracy: Benchmarking Object Detection Models for Pathology Detection in Wireless Capsule Endoscopy

Development and Validation of a Deep Learning-Based Approach to Predict the Mayo Endoscopic Score of Ulcerative Colitis

Modified residual attention network for abnormalities segmentation and detection in WCE images

Lesion2Vec: Deep Metric Learning for Few-Shot Multiple Lesions Recognition in Wireless Capsule Endoscopy Video

Attention Mechanism Guided SE + ResNet-H Model for Gastrointestinal Endoscopy Image Classification

Deep learning for registration of region of interest in consecutive wireless capsule endoscopy frames

Deep convolutional neural networks with ensemble learning and transfer learning for automated detection of gastrointestinal diseases

Gastrointestinal tract disease detection via deep learning based structural and statistical features optimized hexa-classification model

Detection of lymphangiectasia disease from wireless capsule endoscopy images with adaptive threshold

Fusion of Selected Deep CNN and Handcrafted Features for Gastritis Detection from Wireless Capsule Endoscopy Images

Automatic detection of small bowel lesions with different bleeding risks based on deep learning models

Research on a New Feature Detection Algorithm for Wireless Capsule Endoscope Bleeding Images Based on Super-pixel Segmentation.

DFCA-Net: Dual Feature Context Aggregation Network for Bleeding Areas Segmentation in Wireless Capsule Endoscopy Images