Abstract:Background and objective: Feature matching of endoscopic images is of crucial importance in many clinical applications, such as object tracking and surface reconstruction. However, with the presence of low texture, specular reflections and deformations, the feature matching methods of natural scene are facing great challenges in minimally invasive surgery (MIS) scenarios. We propose a novel motion consensus-based method for endoscopic image feature matching to address these problems. Methods: Our method starts by correcting the radial distortion with the spherical projection model and removing the specular reflection regions with an adaptive detection method, which helps to eliminate the image distortion and to reduce the quantity of outliers. We solve the matching problem with a two-stage strategy that progressively estimates a consensus of inliers; the result is a precisely smoothed motion field. First, we construct a spatial motion field from candidate feature matches and estimate its maximum posterior with expectation maximization algorithm, which is computationally efficient and able to obtain smoothed motion field quickly. Second, we extend the smoothed motion field to the affine domain and refine it with bilateral regression to preserve locally subtle motions. The true matches can be identified by checking the difference of feature motion against the estimated field. Results: Evaluations are implemented on two simulation datasets of deformation (218 images) and four different types of endoscopic datasets (1032 images). Our method is compared with three other state-of-the-art methods and achieves the best performance on affine transformation and nonrigid deformation simulations, with inlier ratio of 86.7% and 94.3%, sensitivity of 90.0% and 96.2%, precision of 88.2% and 93.9%, and F1-Score of 89.1% and 95.0%, respectively. On clinical datasets evaluations, the proposed method achieves an average reprojection error of 3.7 pixels and a consistent performance in multi-image correspondence of sequential images. Furthermore, we also present a surface reconstruction result from rhinoscopic images to validate the reliability of our method, which shows high-quality feature matching results. Conclusions: The proposed motion consensus-based feature matching method is proved effective and robust for endoscopic images correspondence. This demonstrates its capability to generate reliable feature matches for surface reconstruction and other meaningful applications in MIS scenarios.

An orientation-free ring feature descriptor with stain-variability normalization for pathology image matching

MRRM: Advanced Biomarker Alignment in Multi-Staining Pathology Images Via Multi-Scale Ring Rotation-Invariant Matching

A Hybrid Deep Feature-Based Deformable Image Registration Method for Pathology Images

MSGM: an Advanced Deep Multi-Size Guiding Matching Network for Whole Slide Histopathology Images Addressing Staining Variation and Low Visibility Challenges

Accurately Matching Serial Whole Slide Images for Tumor Heterogeneity Analysis

Accurate and Robust Alignment of Variable-stained Histologic Images Using a General-purpose Greedy Diffeomorphic Registration Tool

Point-Based Registration for Multi-stained Histology Images

Toward Robust Histology-Prior Embedding for Endomicroscopy Image Classification

Homographic Patch Feature Transform: A Robustness Registration for Gastroscopic Surgery.

Improving Algorithm for the Alignment of Consecutive, Whole-Slide, Immunohistochemical Section Images

Staining Condition Visualization in Digital Histopathological Whole-Slide Images

Accurate and Robust Alignment of Differently Stained Histologic Images Based on Greedy Diffeomorphic Registration

Unpaired Stain Transfer Using Pathology-Consistent Constrained Generative Adversarial Networks

A Deep Learning Method for Automatic Evaluation of Diagnostic Information from Multi-Stained Histopathological Images

Circular Mixture Modeling of Color Distribution for Blind Stain Separation in Pathology Images

Unpaired Stain Style Transfer Using Invertible Neural Networks Based on Channel Attention and Long-Range Residual

Region-aware Registration for Multi-stained Histology Images.

Discovering Clinically Meaningful Shape Features for the Analysis of Tumor Pathology Images

MVFStain: Multiple virtual functional stain histopathology images generation based on specific domain mapping

Automatic Whole Slide Pathology Image Diagnosis Framework Via Unit Stochastic Selection and Attention Fusion

Endoscopic image feature matching via motion consensus and global bilateral regression