Abstract:Background and objective: Functional gastrointestinal disorders (FGIDs) are reported as worldwide gastrointestinal (GI) diseases. GI motility assessment can assist the diagnosis of patients with intestine motility dysfunction. Wireless capsule endoscopy (WCE) can acquire images in the gastrointestinal (GI) tract including the small intestine where other conventional endoscopes cannot penetrate, and WCE images can reveal GI motility. To generally analyze WCE frames, the high-precision registration of consecutive WCE frames is an absolute necessity. It is difficult and meaningless to register entire WCE frames on a pixel level due to the unpredictable and massive non-rigid deformation between consecutive frames, the low quality of imaging and the complex intestinal environment. Thus, the registration of region of interest (ROI) functioning in a feature level has more significance than entire frame registration. Methods: In this paper we present Timecylce-WCE, an end-to-end automatic registration approach of ROIs on WCE images. The clinicians can determine a ROI by drawing a bounding box in any WCE frame to be registered. This proposed approach is based on a deep-learning model of time-consistency in recurrent-registering, skip-registering and self-registering cycle, and it is fully unsupervised without any label. We incorporate the global correlation map with the local correlation map in matching the features, and a novel overall loss function is designed to enable the convergence of the model. As the output, a thin-plate spline (TPS) transformed region in the template frame is highly aligned with the query ROI in a finer-grained level. To the best of our knowledge this is the first time that a deep-learning-based registration method is proposed for WCE imaging motion. Results: To highlight the effectiveness of the proposed approach, our proposed method is compared with the existing non-deep-learning methods and tested in a validation dataset with labeled matching points. The presented method resulted in the best PCK@10 (Percentage of Correct Key-points, i.e., the predicted and the true joint is within the threshold - 10 pixels) of 66.49%. We also demonstrate that variants of design improved registration accuracy. Conclusions: From the experimental analysis, it is clear that our proposed method outperforms the other existing methods. This lays the groundwork for subsequent studies, such as GI motility assessment, and WCE image synthesis.

Recognizing Polyps in Wireless Endoscopy Images Using Deep Stacked Auto Encoder with Constraint Image Model in Flexible Medical Sensor Platform

DeepHealth: Deep Representation Learning with Autoencoders for Healthcare Prediction

An Adaptive Regularization Approach to Colonoscopic Polyp Detection Using a Cascaded Structure of Encoder–Decoders

WCE polyp detection with triplet based embeddings

An Ensemble Framework of Deep Neural Networks for Colorectal Polyp Classification

A Deep Ensemble Learning Method for Colorectal Polyp Classification with Optimized Network Parameters

An automated detection system for colonoscopy images using a dual encoder-decoder model

Wireless Capsule Endoscopy Image Classification Based on Vector Sparse Coding

Modified residual attention network for abnormalities segmentation and detection in WCE images

Deep learning for registration of region of interest in consecutive wireless capsule endoscopy frames

Superpixel Based Segmentation and Classification of Polyps in Wireless Capsule Endoscopy

Advancements in Polyp Detection: A Developed Single Shot Multibox Detector Approach

Modified DeeplabV3+ with multi-level context attention mechanism for colonoscopy polyp segmentation

Multi-Scale Hybrid Network for Polyp Detection in Wireless Capsule Endoscopy and Colonoscopy Images

Stack of discriminative autoencoders for multiclass anomaly detection in endoscopy images

EndoFinder: Online Image Retrieval for Explainable Colorectal Polyp Diagnosis

SSL-CPCD: Self-supervised learning with composite pretext-class discrimination for improved generalisability in endoscopic image analysis

Domain Generalization for Endoscopic Image Segmentation by Disentangling Style-Content Information and SuperPixel Consistency

A real-time deep learning-based system for colorectal polyp size estimation by white-light endoscopy: development and multicenter prospective validation

An Efficient Approach for Polyps Detection in Endoscopic Videos Based on Faster R-CNN

Multi-classification deep learning models for detection of ulcerative colitis, polyps, and dyed-lifted polyps using wireless capsule endoscopy images