Abstract:Background context: The majority of surgical training is conducted in real-world operations. High-fidelity surgical simulators may provide a safer environment for surgical training. However, the extent that it reflects real-world operations and surgical ability is often poorly characterized. Purpose: (1) Assess the validity and fidelity of a surgical simulator; (2) Examine the quantitative relationship between simulation performance and markers of real-world ability; (3) Establish thresholds for surgical expertise, and estimate their external validity and accuracy. Study design/setting: A cohort study of surgeons at a British neurosurgical center. Study sample: 10 early-career "novice" surgeons and 8 board-certified "expert" neurosurgeons. Outcomes measures: (1) Face and content validity, and visual and haptic fidelity; (2) Construct validity; (3) Predictive and discriminative utility of quantitative performance thresholds. Methods: Participants performed unilateral lumbar decompressions on high-fidelity spinal simulators that replicate the bony and soft tissue anatomy along with physiological processes such as bleeding and CSF leaks. Operating times, measured from first surgical action to either self-perceived satisfactory decompression or the end of allocated time, were recorded. The performance was also assessed independently by two blinded spinal subspecialist neurosurgeons using OSATS, a validated surgical assessment tool that utilizes five-point scales on a variety of technical domains to grade the overall technical proficiency. Validity and fidelity were assessed by expert neurosurgeons using quantitative questionnaires. Construct validity was assessed by ordinal regression of simulation performance against real-world surgical grade and portfolio. Thresholds of expert status by simulation performance was established, and their predictive and discriminative utility assessed by crossvalidation accuracy and AUC-ROC. Results: Operating time and expert assessments of simulation performance (OSATS) were strong and significant predictors of surrogate markers of real-world surgical ability. The thresholds for expert status were operating time of 15 minutes and modified OSATS score of 15/20. These thresholds predicted expert status with 84.2% and 71.4% accuracy respectively. Strong discriminative ability was demonstrated by AUC-ROC of 0.95 and 0.83 respectively. All expert surgeons agreed that RealSpine simulators demonstrate high face validity, and high visual and haptic fidelity, with overall scores showing statistically significant agreement on these items (all scores at least 4/5, P<.0001). There was less consensus on content validity, but with still significant overall agreement (average score: 3.75/5, P=0.023). Conclusions: Real-world surgical ability and experience can be accurately predicted by defining objective quantitative thresholds on high-fidelity simulations. The thresholds established here, along with other data presented in this paper, may inform objectives and standards to be established in a spinal surgical training curriculum.

Construct Validity of Simulation Model for Training the Ultrasound-Guided Nerve Block and Catheter Placement

Validity of Multiple-Functioned Periodontal-Surgical Model Used in the Simulation-Based Preclinical Practice Training Course of Periodontology

Learning curves and long-term outcome of simulation-based thoracentesis training for medical students

Characterizing novice behavior associated with learning ultrasound-guided peripheral regional anesthesia

The Effectiveness of Simulation-based Ultrasound-guided Regional Anesthesia Training Programs for Anesthesia Residents: a Prospective Randomized Controlled Trial

Application of simulation-based training in the ultrasound-guided regional anesthesia teaching program

Benefits of simulation for ultrasound-guided midline placement training: MC-in-Sim pilot study

Computer environment for interventional neuroradiology procedures

Three Partial-Task Simulators for Teaching Ultrasound-Guided Regional Anesthesia

Comparison of Learning Curves of Femoral Nerve Block Guided by Nerve Stimulator and Ultrasound for Residents

Comparison of multimodal active learning and single-modality procedural simulation for central venous catheter insertion for incoming residents in anesthesiology: a prospective and randomized study

A tissue-based simulation model for cardiopulmonary bypass cannulation/decannulation training

Use of a Novel Three-Dimensional Model to Teach Ultrasound-guided Subclavian Vein Cannulation

Creation and Piloting of a Model for Simulating a Minimally Invasive Myomectomy

Design and evaluation of UltRASim: An immersive simulator for learning ultrasound-guided regional anesthesia basic skills

An observer tool to enhance learning of incoming anesthesia residents' skills during simulation training of central venous catheter insertion: a randomized controlled trial

Validation of a Novel Non-Biological Bench Model for the Training of Percutaneous Renal Access

Validation of a surgical simulator and establishment of quantitative performance thresholds - RealSpine simulation system for open lumbar decompressions

Simulation-Based Training for Ultrasound-Guided Central Venous Catheter Placement in Pediatric Patients

Development and validation of a synthetic 3D-printed simulator for training in neuroendoscopic ventricular lesion removal

Evaluating the Construct Validity of a Pulsatile Fresh Frozen Human Cadaver Circulation Model for Endovascular Training