Abstract:Purpose: Middle and inner ear procedures target hearing loss, infections, and tumors of the temporal bone and lateral skull base. Despite the advances in surgical techniques, these procedures remain challenging due to limited haptic and visual feedback. Augmented reality (AR) may improve operative safety by allowing the 3D visualization of anatomical structures from preoperative computed tomography (CT) scans on real intraoperative microscope video feed. The purpose of this work was to develop a real-time CT-augmented stereo microscope system using camera calibration and electromagnetic (EM) tracking. Methods: A 3D printed and electromagnetically tracked calibration board was used to compute the intrinsic and extrinsic parameters of the surgical stereo microscope. These parameters were used to establish a transformation between the EM tracker coordinate system and the stereo microscope image space such that any tracked 3D point can be projected onto the left and right images of the microscope video stream. This allowed the augmentation of the microscope feed of a 3D printed temporal bone with its corresponding CT-derived virtual model. Finally, the calibration board was also used for evaluating the accuracy of the calibration. Results: We evaluated the accuracy of the system by calculating the registration error (RE) in 2D and 3D in a microsurgical laboratory setting. Our calibration workflow achieved a RE of 0.11 ± 0.06 mm in 2D and 0.98 ± 0.13 mm in 3D. In addition, we overlaid a 3D CT model on the microscope feed of a 3D resin printed model of a segmented temporal bone. The system exhibited small latency and good registration accuracy. Conclusion: We present the calibration of an electromagnetically tracked surgical stereo microscope for augmented reality visualization. The calibration method achieved accuracy within a range suitable for otologic procedures. The AR process introduces enhanced visualization of the surgical field while allowing depth perception.

From microscope to head-mounted display: integrating hand tracking into microsurgical augmented reality

Augmented reality in the operating room: a clinical feasibility study

Stereoscopic calibration for augmented reality visualization in microscopic surgery

Augmented Reality Microsurgery: Proof of Concept for a Novel Approach to Microsurgical Field Visualization in Plastic Surgery

STTAR: Surgical Tool Tracking Using Off-the-Shelf Augmented Reality Head-Mounted Displays

Spatiotemporal-Aware Augmented Reality: Redefining HCI in Image-Guided Therapy

In Situ Visualization for 3D Ultrasound-Guided Interventions with Augmented Reality Headset

Key Ergonomics Requirements and Possible Mechanical Solutions for Augmented Reality Head-Mounted Displays in Surgery

Robust hand tracking for surgical telestration

The potentials for hands-free interaction in micro-neurosurgery

Closing the Calibration Loop: An Inside-out-tracking Paradigm for Augmented Reality in Orthopedic Surgery

An augmented reality and high-speed optical tracking system for laparoscopic surgery

Feasibility of using a low-cost head-mounted augmented reality device in the operating room

Augmented reality-based navigation system for wrist arthroscopy: feasibility

Reflective-AR Display: An Interaction Methodology for Virtual-Real Alignment in Medical Robotics

A Comparative Evaluation of Optical See-Through Augmented Reality in Surgical Guidance.

Co-localized augmented human and X-ray observers in collaborative surgical ecosystem

An integrated approach to endoscopic instrument tracking for augmented reality applications in surgical simulation training

Robotic Manipulator‐Assisted Omnidirectional Augmented Reality for Endoluminal Intervention Telepresence

Real-time Geometry-Aware Augmented Reality in Minimally Invasive Surgery

Integration of Augmented Reality in Temporal Bone and Skull Base Surgeries