Abstract:Background and objective: Feature matching of endoscopic images is of crucial importance in many clinical applications, such as object tracking and surface reconstruction. However, with the presence of low texture, specular reflections and deformations, the feature matching methods of natural scene are facing great challenges in minimally invasive surgery (MIS) scenarios. We propose a novel motion consensus-based method for endoscopic image feature matching to address these problems. Methods: Our method starts by correcting the radial distortion with the spherical projection model and removing the specular reflection regions with an adaptive detection method, which helps to eliminate the image distortion and to reduce the quantity of outliers. We solve the matching problem with a two-stage strategy that progressively estimates a consensus of inliers; the result is a precisely smoothed motion field. First, we construct a spatial motion field from candidate feature matches and estimate its maximum posterior with expectation maximization algorithm, which is computationally efficient and able to obtain smoothed motion field quickly. Second, we extend the smoothed motion field to the affine domain and refine it with bilateral regression to preserve locally subtle motions. The true matches can be identified by checking the difference of feature motion against the estimated field. Results: Evaluations are implemented on two simulation datasets of deformation (218 images) and four different types of endoscopic datasets (1032 images). Our method is compared with three other state-of-the-art methods and achieves the best performance on affine transformation and nonrigid deformation simulations, with inlier ratio of 86.7% and 94.3%, sensitivity of 90.0% and 96.2%, precision of 88.2% and 93.9%, and F1-Score of 89.1% and 95.0%, respectively. On clinical datasets evaluations, the proposed method achieves an average reprojection error of 3.7 pixels and a consistent performance in multi-image correspondence of sequential images. Furthermore, we also present a surface reconstruction result from rhinoscopic images to validate the reliability of our method, which shows high-quality feature matching results. Conclusions: The proposed motion consensus-based feature matching method is proved effective and robust for endoscopic images correspondence. This demonstrates its capability to generate reliable feature matches for surface reconstruction and other meaningful applications in MIS scenarios.

A Robust Mosaicing Method with Super-Resolution for Optical Medical Images.

Robust Mosaicing of Endomicroscopic Videos Via Context-Weighted Correlation Ratio.

Robust Endoscopic Image Mosaicking Via Fusion of Multimodal Estimation

Medical Image Mosaic Based On Low-Overlapping Regions

A Global Approach for Automatic Fibroscopic Video Mosaicing in Minimally Invasive Diagnosis

Novel OCT mosaicking pipeline with Feature- and Pixel-based registration

Mosaicing the Retinal Fundus Images: a Robust Registration Technique Based Approach

Image Mosaicking for Oversized Documents with a Multi-Camera Rig

Robust Subpixel Registration For Image Mosaicing

A self-supervised and adversarial approach to hyperspectral demosaicking and RGB reconstruction in surgical imaging

Accurate and robust feature description and dense point-wise matching based on feature fusion for endoscopic images

A robust mosaicking procedure for high spatial resolution remote sensing images

Super Resolution in Robotic-Assisted Minimally Invasive Surgery.

Mosaic of medical ultrasound image based on adaptive simulated annealing and multiresolution searching

Multi-view Multi-modal Feature Embedding for Endomicroscopy Mosaic Classification

Convolutional Neural Network-Based Registration for Mosaicing of Microscopic Images

Computational framework for generating large panoramic super-resolution images from localization microscopy

Camera Image Mosaicing Based on an Optimized SURF Algorithm

Endoscopic image feature matching via motion consensus and global bilateral regression

Deep learning approach for hyperspectral image demosaicking, spectral correction and high-resolution RGB reconstruction

From Macro to Micro: Autonomous Multiscale Image Fusion for Robotic Surgery.