Abstract:Surgical navigation systems involve various technologies of segmentation, calibration, registration, tracking, and visualization. These systems aim to superimpose multisource information in the surgical field and provide surgeons with a composite overlay (augmented-reality) view, improving the operative precision and experience. Surgical 3-D tracking is the key to build these systems. Unfortunately, surgical 3-D tracking is still a challenge to endoscopic and robotic navigation systems and easily gets trapped in image artifacts, tissue deformation, and inaccurate positional (e.g., electromagnetic) sensor measurements. This work explores a new monocular endoscope hybrid 3-D tracking method called spatially constrained adaptive differential evolution that combines two spatial constraints with observation-recall adaptive propagation and observation-based fitness computing for stochastic optimization. Specifically, we spatially constraint inaccurate electromagnetic sensor measurements to the centerline of anatomical tubular structures to keep them physically locating inside the tubes, as well as interpolate these measurements to reduce jitter errors for smooth 3-D tracking. We then propose observation-recall adaptive propagation with fitness computing to precisely fuse the constrained sensor measurements, preoperative images, and endoscopic video sequences for accurate hybrid 3-D tracking. Additionally, we also propose a new marker-free hybrid registration strategy to precisely align positional sensor measurements to preoperative images. Our new framework was evaluated on a large amount of clinical data acquired from various surgical endoscopic procedures, with the experimental results showing that it certainly outperforms current surgical 3-D approaches. In particular, the position and rotation errors were significantly reduced from (6.55, 11.4) to (3.02 mm, 8.54 °).

Endoscope tracking and positioning and digital human dynamic synchronous display device based on multi-sensor

A Multimodal Sensing CMOS Imager Based on Dual‐Focus Imaging(Adv. Sci. 14/2023)

Synchronous imaging control system for two modes of cone beam X-ray luminescence CT and CT

Multisource Differential Fusion Driven Monocular Endoscope Hybrid 3-D Tracking for Advanced Endoscopic Navigation Surgery

Motion-Guided Dual-Camera Tracker for Endoscope Tracking and Motion Analysis in a Mechanical Gastric Simulator

On the Magnetic Localization Techniques for Position and Orientation of the Capsule Endoscope

A Flexible Attitude System for Wireless Micro-Ball Endoscopy

The Software Implementation of Magnetic Localization and Orientation Detection for Capsule Endoscope

A New Calibration Method For Magnetic Sensor Array For Tracking Capsule Endoscope

Multipoint Simultaneous Tracking of Wireless Capsule Endoscope Using Magnetic Sensor Array

3D Position Tracking using On-chip Magnetic Sensing in Image-guided Navigation Bronchoscopy

A Cubic 3-Axis Magnetic Sensor Array for Wirelessly Tracking Magnet Position and Orientation

Design of Bi-Directional, Digital and Miniaturized Wireless Endoscopy System

Two-Magnet-Based 6d-Localization And Orientation For Wireless Capsule Endoscope

Design of A Data Acquisition System on Magnetic Signal for Magnetic Localization and Orientation System

Active Actuation System of Wireless Capsule Endoscope Based on Magnetic Field.

An EM-Tracked Approach for Calibrating the 3D Pose of Flexible Endoscopes

Freehand 3D Ultrasound Imaging Based on Probe-mounted Vision and IMU System

A Localization Method Using 3-Axis Magnetoresistive Sensors for Tracking of Capsule Endoscope.

A New Tracking System for Three Magnetic Objectives

Micro Measurement System for Tracking Human Body Motion