Abstract:<p>Current capsule endoscopes and next-generation robotic capsules for diagnosis and treatment of gastrointestinal diseases are complex cyber-physical platforms that must orchestrate complex software and hardware functions. The desired tasks for these systems include visual localization, depth estimation, 3D mapping, disease detection and segmentation, automated navigation, active control, path realization and optional therapeutic modules such as targeted drug delivery and biopsy sampling. Data-driven algorithms promise to enable many advanced functionalities for capsule endoscopes, but real-world data is challenging to obtain. Physically-realistic simulations providing synthetic data have emerged as a solution to the development of data-driven algorithms. In this work, we present a comprehensive simulation platform for capsule endoscopy operations and introduce VR-Caps, a virtual active capsule environment that simulates a range of normal and abnormal tissue conditions (e.g., inflated, dry, wet etc.) and varied organ types, capsule endoscope designs (e.g., mono, stereo, dual and 360<span class="math"><math>∘</math></span> camera), and the type, number, strength, and placement of internal and external magnetic sources that enable active locomotion. VR-Caps makes it possible to both independently or jointly develop, optimize, and test medical imaging and analysis software for the current and next-generation endoscopic capsule systems. To validate this approach, we train state-of-the-art deep neural networks to accomplish various medical image analysis tasks using simulated data from VR-Caps and evaluate the performance of these models on real medical data. Results demonstrate the usefulness and effectiveness of the proposed virtual platform in developing algorithms that quantify fractional coverage, camera trajectory, 3D map reconstruction, and disease classification. All of the code, pre-trained weights and created 3D organ models of the virtual environment with detailed instructions how to setup and use the environment are made publicly available at <a href="https://github.com/CapsuleEndoscope/VirtualCapsuleEndoscopy">https://github.com/CapsuleEndoscope/VirtualCapsuleEndoscopy</a> and a video demonstration can be seen in the supplementary videos (Video-I).</p>

Capsule Vision 2024 Challenge: Multi-Class Abnormality Classification for Video Capsule Endoscopy

Multi-Class Abnormality Classification Task in Video Capsule Endoscopy

Multi-Class Abnormality Classification in Video Capsule Endoscopy Using Deep Learning

Video Capsule Endoscopy Classification using Focal Modulation Guided Convolutional Neural Network

Capsule Endoscopy Multi-classification via Gated Attention and Wavelet Transformations

Kvasir-Capsule, a video capsule endoscopy dataset

The intersection of video capsule endoscopy and artificial intelligence: addressing unique challenges using machine learning

A Hybrid Convolutional Neural Network with Meta Feature Learning for Abnormality Detection in Wireless Capsule Endoscopy Images

CASCRNet: An Atrous Spatial Pyramid Pooling and Shared Channel Residual based Network for Capsule Endoscopy

A Multimodal Approach For Endoscopic VCE Image Classification Using BiomedCLIP-PubMedBERT

Reduction of Video Capsule Endoscopy Reading Times Using Deep Learning with Small Data

ViTCA-Net: a framework for disease detection in video capsule endoscopy images using a vision transformer and convolutional neural network with a specific attention mechanism

CapsuleNet: A Deep Learning Model To Classify GI Diseases Using EfficientNet-b7

PS-DeVCEM: Pathology-sensitive deep learning model for video capsule endoscopy based on weakly labeled data

Classification of Endoscopy and Video Capsule Images using CNN-Transformer Model

VR-Caps: A Virtual Environment for Capsule Endoscopy

Integrating Deep Feature Extraction and Hybrid ResNet-DenseNet Model for Multi-Class Abnormality Detection in Endoscopic Images

Graph Convolution Neural Network For Weakly Supervised Abnormality Localization In Long Capsule Endoscopy Videos

GastroVision: A Multi-class Endoscopy Image Dataset for Computer Aided Gastrointestinal Disease Detection

Domain-Adaptive Pre-training of Self-Supervised Foundation Models for Medical Image Classification in Gastrointestinal Endoscopy

FUSECAPS: Investigating Feature Fusion Based Framework for Capsule Endoscopy Image Classification