Abstract:Surgical navigation systems involve various technologies of segmentation, calibration, registration, tracking, and visualization. These systems aim to superimpose multisource information in the surgical field and provide surgeons with a composite overlay (augmented-reality) view, improving the operative precision and experience. Surgical 3-D tracking is the key to build these systems. Unfortunately, surgical 3-D tracking is still a challenge to endoscopic and robotic navigation systems and easily gets trapped in image artifacts, tissue deformation, and inaccurate positional (e.g., electromagnetic) sensor measurements. This work explores a new monocular endoscope hybrid 3-D tracking method called spatially constrained adaptive differential evolution that combines two spatial constraints with observation-recall adaptive propagation and observation-based fitness computing for stochastic optimization. Specifically, we spatially constraint inaccurate electromagnetic sensor measurements to the centerline of anatomical tubular structures to keep them physically locating inside the tubes, as well as interpolate these measurements to reduce jitter errors for smooth 3-D tracking. We then propose observation-recall adaptive propagation with fitness computing to precisely fuse the constrained sensor measurements, preoperative images, and endoscopic video sequences for accurate hybrid 3-D tracking. Additionally, we also propose a new marker-free hybrid registration strategy to precisely align positional sensor measurements to preoperative images. Our new framework was evaluated on a large amount of clinical data acquired from various surgical endoscopic procedures, with the experimental results showing that it certainly outperforms current surgical 3-D approaches. In particular, the position and rotation errors were significantly reduced from (6.55, 11.4) to (3.02 mm, 8.54 °).

Endoscopes Shape Reconstruction Based on Electromagnetic Localization and Curve Fitting.

Three-Dimensional Stitching of Binocular Endoscopic Images Based on Feature Points

Trans-scale Optical Endoscopy Imaging Technology

A Three-Dimensional Measurement Method for Binocular Endoscopes Based on Deep Learning

Design of High Resolution Panoramic Endoscope Imaging System Based on Freeform Surface

Endoscopic Optical Coherence Tomography Angiography Using Inverse SNR-amplitude Decorrelation Features and Electrothermal Micro-Electro-mechanical System Raster Scan

A Locally Weighted Method for Electromagnetic Localization of Endoscopes

An EM-Tracked Approach for Calibrating the 3D Pose of Flexible Endoscopes

Wearable Electromagnetic Apparatus for Endoscope Localization

Incorporation of 3D Model and Panoramic View for Gastroscopic Lesion Surveillance.

Motor-free telerobotic endomicroscopy for steerable and programmable imaging in complex curved and localized areas

Magnetic-Visual Sensor Fusion-based Dense 3D Reconstruction and Localization for Endoscopic Capsule Robots

The Software Implementation of Magnetic Localization and Orientation Detection for Capsule Endoscope

An Improved Magnetic Localization And Orientation Algorithm For Wireless Capsule Endoscope

2D Reconstruction of Small Intestine's Interior Wall

A Localization Method Using 3-Axis Magnetoresistive Sensors for Tracking of Capsule Endoscope.

Multisource Differential Fusion Driven Monocular Endoscope Hybrid 3-D Tracking for Advanced Endoscopic Navigation Surgery

Multi-Interface Strain Transfer Modeling for Flexible Endoscope Shape Sensing

Multistage adaptive control strategy based on image contour data for autonomous endoscope navigation

Design, Analysis, and Kinematic Framework of a Long-Range Magnetic Anchored and Guided Endoscope

Photometric single-view dense 3D reconstruction in endoscopy