Abstract:Background context: The majority of surgical training is conducted in real-world operations. High-fidelity surgical simulators may provide a safer environment for surgical training. However, the extent that it reflects real-world operations and surgical ability is often poorly characterized. Purpose: (1) Assess the validity and fidelity of a surgical simulator; (2) Examine the quantitative relationship between simulation performance and markers of real-world ability; (3) Establish thresholds for surgical expertise, and estimate their external validity and accuracy. Study design/setting: A cohort study of surgeons at a British neurosurgical center. Study sample: 10 early-career "novice" surgeons and 8 board-certified "expert" neurosurgeons. Outcomes measures: (1) Face and content validity, and visual and haptic fidelity; (2) Construct validity; (3) Predictive and discriminative utility of quantitative performance thresholds. Methods: Participants performed unilateral lumbar decompressions on high-fidelity spinal simulators that replicate the bony and soft tissue anatomy along with physiological processes such as bleeding and CSF leaks. Operating times, measured from first surgical action to either self-perceived satisfactory decompression or the end of allocated time, were recorded. The performance was also assessed independently by two blinded spinal subspecialist neurosurgeons using OSATS, a validated surgical assessment tool that utilizes five-point scales on a variety of technical domains to grade the overall technical proficiency. Validity and fidelity were assessed by expert neurosurgeons using quantitative questionnaires. Construct validity was assessed by ordinal regression of simulation performance against real-world surgical grade and portfolio. Thresholds of expert status by simulation performance was established, and their predictive and discriminative utility assessed by crossvalidation accuracy and AUC-ROC. Results: Operating time and expert assessments of simulation performance (OSATS) were strong and significant predictors of surrogate markers of real-world surgical ability. The thresholds for expert status were operating time of 15 minutes and modified OSATS score of 15/20. These thresholds predicted expert status with 84.2% and 71.4% accuracy respectively. Strong discriminative ability was demonstrated by AUC-ROC of 0.95 and 0.83 respectively. All expert surgeons agreed that RealSpine simulators demonstrate high face validity, and high visual and haptic fidelity, with overall scores showing statistically significant agreement on these items (all scores at least 4/5, P<.0001). There was less consensus on content validity, but with still significant overall agreement (average score: 3.75/5, P=0.023). Conclusions: Real-world surgical ability and experience can be accurately predicted by defining objective quantitative thresholds on high-fidelity simulations. The thresholds established here, along with other data presented in this paper, may inform objectives and standards to be established in a spinal surgical training curriculum.

Assessment of a percutaneous iliosacral screw insertion simulator

Assessment of a percutaneous iliosacral screw insertion simulator

Improving surgical technical skills for emergency fixation of unstable pelvic ring fractures: an experimental study using a pelvic ring fracture simulator

Accuracy of navigated and conventional iliosacral screw placement in B- and C-type pelvic ring fractures

Toward the next generation of simulator for intraoperative navigation of scoliotic spine surgeries

Sacroiliac joint fusion guided by intraoperatively superimposed virtual surgical planning using simulated fluoroscopic images

Image guided interpedicular screw placement simulation system for training and skill evaluation. Proof of concept

Precision Insertion of Percutaneous Sacroiliac Screws Using a Novel Augmented Reality-Based Navigation System: a Pilot Study

Comparison of Computer‐Assisted Navigation and 3D Printed Patient‐Specific Template for the Iliosacral Screw Placement

The Effectiveness of Bioskills Training for Simulated Lumbar Pedicle Screw Placement

A cadaveric precision and accuracy analysis of augmented reality-mediated percutaneous pedicle implant insertion

Multi-institutional beta testing of a novel cervical esophagogastric anastomosis simulator

Surgical skill simulation training to proficiency reduces procedural errors among novice cardiac device implanters: a randomized study

Simplifying the Orientation of S1 Iliosacral Screws for Placement in the Dysmorphic Sacrum

Percutaneous thoraco-lumbar-sacral pedicle screw placement accuracy results from a multi-center, prospective clinical study using a skin marker-based optical navigation system

Validation of a surgical simulator and establishment of quantitative performance thresholds - RealSpine simulation system for open lumbar decompressions

2D versus 3D fluoroscopy-based navigation in posterior pelvic fixation: review of the literature on current technology

Fluoroscopy-based navigation system in spine surgery

Accuracy of augmented reality surgical navigation for minimally invasive pedicle screw insertion in the thoracic and lumbar spine with a new tracking device

Qualitative and Quantitative Assessment of Invasive Surgery Skill Based on a Custom Training Simulator

Percutaneous Sacroiliac Screw Fixation: A Modified Screw Insertion Method Using Just 2 Fluoroscopy Views