Abstract:Purpose: As the trend toward minimally invasive and percutaneous interventions continues, the importance of appropriate surgical data visualization becomes more evident. Ineffective interventional data display techniques that yield poor ergonomics that hinder hand-eye coordination, and therefore promote frustration which can compromise on-task performance up to adverse outcome. A very common example of ineffective visualization is monitors attached to the base of mobile C-arm X-ray systems. Methods: We present a spatially and imaging geometry-aware paradigm for visualization of fluoroscopic images using Interactive Flying Frustums (IFFs) in a mixed reality environment. We exploit the fact that the C-arm imaging geometry can be modeled as a pinhole camera giving rise to an 11-degree-of-freedom view frustum on which the X-ray image can be translated while remaining valid. Visualizing IFFs to the surgeon in an augmented reality environment intuitively unites the virtual 2D X-ray image plane and the real 3D patient anatomy. To achieve this visualization, the surgeon and C-arm are tracked relative to the same coordinate frame using image-based localization and mapping, with the augmented reality environment being delivered to the surgeon via a state-of-the-art optical see-through head-mounted display. Results: The root-mean-squared error of C-arm source tracking after hand-eye calibration was determined as [Formula: see text] and [Formula: see text] in rotation and translation, respectively. Finally, we demonstrated the application of spatially aware data visualization for internal fixation of pelvic fractures and percutaneous vertebroplasty. Conclusion: Our spatially aware approach to transmission image visualization effectively unites patient anatomy with X-ray images by enabling spatial image manipulation that abides image formation. Our proof-of-principle findings indicate potential applications for surgical tasks that mostly rely on orientational information such as placing the acetabular component in total hip arthroplasty, making us confident that the proposed augmented reality concept can pave the way for improving surgical performance and visuo-motor coordination in fluoroscopy-guided surgery.

Intra-Operative Visualizations: Perceptual Fidelity and Human Factors

Augmented reality in the operating room: a clinical feasibility study

Augmented Reality Computational Spectral Imaging for Surgical Guidance

Interactive Flying Frustums (IFFs): spatially aware surgical data visualization

Spatiotemporal-Aware Augmented Reality: Redefining HCI in Image-Guided Therapy

Visualization of surgical maneuvers using intraoperative real-time volumetric optical coherence tomography

High-Quality See-Through Surgical Guidance System Using Enhanced 3-D Autostereoscopic Augmented Reality

3D Visualization and Augmented Reality for Orthopedics.

3D Interactive Surgical Visualization System Using Mobile Spatial Information Acquisition and Autostereoscopic Display

Augmented reality-based autostereoscopic surgical visualization system for telesurgery

A Comparative Evaluation of Optical See-Through Augmented Reality in Surgical Guidance.

A High-Accuracy Surgical Augmented Reality System Using Enhanced Integral Videography Image Overlay.

On-the-fly Augmented Reality for Orthopaedic Surgery Using a Multi-Modal Fiducial

Visualization, Registration and Tracking Techniques for Augmented Reality Guided Surgery: a Review

In Situ Visualization for 3D Ultrasound-Guided Interventions with Augmented Reality Headset

8.2: Anatomy Education Method Using Autostereoscopic 3D Image Overlay and Mid‐Air Augmented Reality Interaction

[Primary pulmonary manifestation of extramedullary acute myelocytic leukemia].

A novel hybrid technology for remediation of molasses-based raw effluents.

Real-Time Computer-Generated Integral Imaging and 3d Image Calibration for Augmented Reality Surgical Navigation

Toward Intraoperative Visual Intelligence: Real-Time Surgical Instrument Segmentation for Enhanced Surgical Monitoring