Abstract:Purpose: Middle and inner ear procedures target hearing loss, infections, and tumors of the temporal bone and lateral skull base. Despite the advances in surgical techniques, these procedures remain challenging due to limited haptic and visual feedback. Augmented reality (AR) may improve operative safety by allowing the 3D visualization of anatomical structures from preoperative computed tomography (CT) scans on real intraoperative microscope video feed. The purpose of this work was to develop a real-time CT-augmented stereo microscope system using camera calibration and electromagnetic (EM) tracking. Methods: A 3D printed and electromagnetically tracked calibration board was used to compute the intrinsic and extrinsic parameters of the surgical stereo microscope. These parameters were used to establish a transformation between the EM tracker coordinate system and the stereo microscope image space such that any tracked 3D point can be projected onto the left and right images of the microscope video stream. This allowed the augmentation of the microscope feed of a 3D printed temporal bone with its corresponding CT-derived virtual model. Finally, the calibration board was also used for evaluating the accuracy of the calibration. Results: We evaluated the accuracy of the system by calculating the registration error (RE) in 2D and 3D in a microsurgical laboratory setting. Our calibration workflow achieved a RE of 0.11 ± 0.06 mm in 2D and 0.98 ± 0.13 mm in 3D. In addition, we overlaid a 3D CT model on the microscope feed of a 3D resin printed model of a segmented temporal bone. The system exhibited small latency and good registration accuracy. Conclusion: We present the calibration of an electromagnetically tracked surgical stereo microscope for augmented reality visualization. The calibration method achieved accuracy within a range suitable for otologic procedures. The AR process introduces enhanced visualization of the surgical field while allowing depth perception.

Registration of preoperative temporal bone CT-scan to otoendoscopic video for augmented-reality based on convolutional neural networks

Monocular Microscope to CT Registration using Pose Estimation of the Incus for Augmented Reality Cochlear Implant Surgery

Early Feasibility Studies of Augmented Reality Navigation for Lateral Skull Base Surgery

A practical marker-less image registration method for augmented reality oral and maxillofacial surgery

Stereoscopic calibration for augmented reality visualization in microscopic surgery

A novel augmented reality system for displaying inferior alveolar nerve bundles in maxillofacial surgery

Augmented Reality Navigation with Automatic Marker-Free Image Registration Using 3-D Image Overlay for Dental Surgery

Vision-based Markerless Registration Using Stereo Vision and an Augmented Reality Surgical Navigation System: a Pilot Study.

Augmented Reality Registration System for Visualization of Skull Landmarks

Calibration of RGBD camera and cone-beam CT for 3D intra-operative mixed reality visualization

Monocular Endoscope Video-based Augmented Reality for Transoral Laryngeal Tumor Resection Surgery

Fusion of Augmented Reality Imaging with the Endoscopic View for Endonasal Skull Base Surgery; a Novel Application for Surgical Navigation Based on Intraoperative Cone Beam Computed Tomography and Optical Tracking

Integration of Augmented Reality in Temporal Bone and Skull Base Surgeries

Scale-invariant registration of monocular endoscopic images to CT-scans for sinus surgery

2D/3D registration and motion tracking for surgical interventions

Validation of a novel AI-based automated multimodal image registration of CBCT and intraoral scan aiding presurgical implant planning

Application of preoperative registration and automatic tracking technique for image-guided maxillofacial surgery

A spatial registration method based on 2D-3D registration for an augmented reality spinal surgery navigation system

A spatial registration method for navigation system combining O-arm with spinal surgery robot

Automatic Registration and Error Color Maps to Improve Accuracy for Navigated Bone Tumor Surgery Using Intraoperative Cone-Beam CT

Real-Time Marker-Free Patient Registration and Image-Based Navigation Using Stereovision for Dental Surgery.