Unsupervised Neural Network-Based Image Stitching Method for Bladder Endoscopy

Ye Zixing,Shao Chenyu,Zhu Kelei

DOI: https://doi.org/10.1101/2024.09.24.614700

2024-09-25

Abstract:Bladder endoscopy enables the observation of intravesical lesion characteristics, making it an essential tool in urology. Image stitching techniques are commonly employed to expand the field of view of bladder endoscopy. Traditional image stitching methods rely on feature matching. In recent years, deep-learning techniques have garnered significant attention in the field of computer vision. However, the commonly employed supervised learning approaches often require a substantial amount of labeled data, which can be challenging to acquire, especially in the context of medical data. Both feature-based and unreliable supervised methods for cystoscopy image stitching are limited by their quality and the robustness of image stitching. This study proposes an unsupervised neural network-based image stitching method for bladder endoscopy that comprises two modules: an unsupervised alignment network and an unsupervised fusion network. In the unsupervised alignment network, we employed feature convolution, regression networks, and linear transformations to align images. In the unsupervised fusion network, we achieved image fusion from features to pixel by simultaneously eliminating artifacts and enhancing the resolution. Experiments demonstrated our method's consistent stitching success rate of 98.11% and robust image stitching accuracy at various resolutions. Our method eliminates sutures and flocculent debris from cystoscopy images, presenting good image smoothness while preserving rich textural features. Moreover, our method could successfully stitch challenging scenes such as dim and blurry scenes. Our application of unsupervised deep learning methods in the field of cystoscopy image stitching was successfully validated, laying the foundation for real-time panoramic stitching of bladder endoscopic video images. This advancement provides opportunities for the future development of computer-vision-assisted diagnostic systems for bladder cavities.

Bioengineering

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to address the issues in bladder endoscopic image stitching. Specifically: 1. **Background and Challenges**: - Bladder endoscopy is a crucial tool for diagnosing and monitoring bladder cancer, but due to limited field of view, image stitching technology becomes particularly important. - Traditional image stitching methods rely on feature matching, while supervised learning-based methods require a large amount of annotated data, which is difficult to obtain in the medical field. - Existing feature-based and unreliable supervised methods have limited quality and robustness in bladder endoscopic image stitching. 2. **Research Objectives**: - Propose an unsupervised neural network image stitching method for seamless stitching of bladder endoscopic images. - This method includes two modules: an unsupervised alignment network and an unsupervised fusion network. - In the unsupervised alignment network, images are aligned through feature convolution, regression network, and linear transformation. - In the unsupervised fusion network, image fusion from feature to pixel level is achieved by simultaneously eliminating artifacts and enhancing resolution. 3. **Main Contributions**: - Experimental results show that the success rate of stitching with this method remains stable at 98.11% under different resolutions, demonstrating high robustness. - This method can effectively eliminate seams and artifacts in bladder endoscopic images, presenting good image smoothness while retaining rich texture features. - Especially in dim and blurry scenes, this method can also successfully perform stitching. 4. **Application Prospects**: - This study validates the effectiveness of unsupervised deep learning methods in the field of bladder endoscopic image stitching, laying the foundation for real-time panoramic stitching. - This advancement provides opportunities for the development of future computer vision-assisted bladder cavity diagnostic systems.

Unsupervised Neural Network-Based Image Stitching Method for Bladder Endoscopy

Three-Dimensional Stitching of Binocular Endoscopic Images Based on Feature Points

Gastrointestinal image stitching based on improved unsupervised algorithm

Deep Learning Based Cystoscopy Image Enhancement

Unsupervised Deep Image Stitching: Reconstructing Stitched Features to Images

Self-Supervised Monocular Depth Estimation for Endoscopic Imaging

Application of artificial neural networks for automated analysis of cystoscopic images: a review of the current status and future prospects

U-Net-Based Assistive Identification of Bladder Cancer: A Promising Approach for Improved Diagnosis

Self-supervised neural network-based endoscopic monocular 3D reconstruction method

A transfer-learning approach for lesion detection in endoscopic images from the urinary tract

SynStitch: a Self-Supervised Learning Network for Ultrasound Image Stitching Using Synthetic Training Pairs and Indirect Supervision

Using spatial-temporal ensembles of convolutional neural networks for lumen segmentation in ureteroscopy

Intelligent Inspection Guidance of Urethral Endoscopy Based on SLAM with Blood Vessel Attentional Features

A comparative study of attention mechanism based deep learning methods for bladder tumor segmentation

Automatic Image Unfolding and Stitching Framework for Esophageal Lining Video Based on Density-Weighted Feature Matching

Percutaneous Nephrostomy Guidance by a Convolutional Neural Network Based Endoscopic Optical Coherence Tomography System

2D and 3D Bladder Segmentation Using U-Net-based Deep-Learning

Comparative study of image registration techniques for bladder video-endoscopy

Automatic bladder segmentation from CT images using deep CNN and 3D fully connected CRF-RNN

Bladder Wall Segmentation in MRI Images Via Deep Learning and Anatomical Constraints.

Self-supervised monocular depth estimation for gastrointestinal endoscopy