Abstract:Introduction A team of biomedical engineers, neurosurgeons, and a medical educator, created and evaluated preliminary validity evidence of a high-fidelity simulator used to train ventriculostomy skills. Description A scale reproduction of an adult skull and relevant anatomy was created. Twelve neurosurgery residents, one fellow, and five attendings (n=17) from three academic medical centers performed the simulated creation of a ventricolostomy. Participants rated the simulator using a 37-item survey across five domains. Quality of the simulator was scored from “Not at all realistic” (1) to “Highly realistic” (4). Self-reported ability to perform relevant tasks was rated from “Too difficult to perform (0) to “Too easy to perform” (4). Validity evidence relevant to test content was evaluated using a Rasch model, while evidence relevant to internal structure (inter-item reliability, and inter-rater agreement) was evaluated using traditional methods. Conclusion Analyses indicated attendings had statistically higher ratings (M=3.4/4.0) than fellows (M=3.3), and residents (M=3.0), p=.02. Domain means were 3.9 (Value), 3.5 (Physical attributes), 3.4 (Realism of experience), 3.3 (Relevance), and 2.9 (Ability). Inter-item consistency across domains were estimated to be moderate-high (.65, .92), and inter-rater agreement regarding simulator characteristics was high [ICC(2,k)= .90]. Rating differences were found across institutions, p=.001, and are reviewed. The observed global rating (2.5) indicated the simulator can be considered for teaching ventriculostomies, but could be improved slightly. Ratings indicated the simulator was valuable as a learning tool, but could be improved with minor modifications. The most commonly-suggested improvement was the addition of ears as an added anatomical landmark. References 1. Haji FA, Dubrowski A, Drake J, de Ribaupierre S. Needs assessment for simulation training in neuroendoscopy: a Canadian national survey. J Neurosurg. 2013 Feb;118(2):250-7. doi: 10.3171/2012.10.JNS12767. Epub 2012 Dec 7 2. Schirmer CM, Elder JB, Roitberg B, Lobel DA. Virtual reality-based simulation training for ventriculostomy: an evidence-based approach. Neurosurgery. 2013 Oct;73 Suppl 1:66-73. doi: 10.1227/NEU.0000000000000074. 3. Choudhury N, Gélinas-Phaneuf N, Delorme S, Del Maestro R. Fundamentals of neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg. 2013 Nov;80(5):e9-19. doi: 10.1016/j.wneu.2012.08.022. Epub 2012 Nov 23. 4. Korndorffer JR Jr, Kasten SJ, Downing SM. A call for the utilization of consensus standards in the surgical education literature. Am J Surg. 2010 Jan;199(1):99-104. doi: 10.1016/j.amjsurg.2009.08.018 5. Cook DA, Brydges R, Zendejas B, Hamstra SJ, Hatala R. Technology-enhanced simulation to assess health professionals: A systematic review of validity evidence, research methods, and reporting quality. Acad Med. 2013 Jun;88(6):872-83. doi: 10.1097/ACM.0b013e31828ffdcf. 6. Cook DA, Zendejas B, Hamstra SJ, Hatala R, Brydges R. What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment. Adv Health Sci Educ Theory Pract. 2014 May;19(2):233-50. doi: 10.1007/s10459-013-9458-4. Epub 2013 May 2. 7. Standards for Educational and Psychological Testing, 1999, American Educational Research Association, American Psychological Association and National Council on Measurement in Education: American Educational Research. 8. Rasch G. (1960/1980). Probabilistic models for some intelligence and attainment tests. (Copenhagen, Danish Institute for Educational Research), expanded edition (1980) with foreword and afterword by BD Wright. Chicago, IL: The University of Chicago Press. 9. Hamilton JM, et al., Toward effective pediatric minimally invasive surgical simulation. J Pediatr Surg, 2011. 46(1): p. 138-44. Disclosures None

Collating evidence to support the validation of a simulated laparotomy incision and closure-training model

Validity and reliability evidence support task-specific metrics for laparoscopic fundoplication

Development and validation of a simulation-based assessment of operative competence for higher specialist trainees in general surgery

Using Artificial Intelligence to Gauge Competency on a Novel Laparoscopic Training System

Design and validation of an assessment tool for open surgical procedures

Validation of a virtual intracorporeal suturing simulator

Development and Validation of a Laparoscopic Sacrocolpopexy Training Model

Trial of an Inexpensive Training Simulation Model for Laparoscopic Appendicectomy

Using an application to measure trainees’ procedural knowledge before chest tube insertion

Training validity of laparoscopic virtual reality simulator

Evaluating the Validity of National Multiassessment System in Postgraduate Surgical Training: A Retrospective Cohort Study

Creation and Piloting of a Model for Simulating a Minimally Invasive Myomectomy

Validation of a Simulation Model for Robotic Myomectomy

Anastomosis Lapse Index (ALI): A Validated End Product Assessment Tool for Simulation Microsurgery Training

Current status of simulation and training models in microsurgery: A systematic review

Board #322 - Technology Innovation Development and Preliminary Validation of a Novel Ventriculostomy Simulator (Submission #8635)

Multi-institutional beta testing of a novel cervical esophagogastric anastomosis simulator

Assessing the Efficacy of Enhanced Versus Standard Three-Dimensional Training Models in Laparoscopic Skills Acquisition: A Randomized Controlled Trial of Novice Medical Students

Development and clinical implementation of a structured, simulation-based training programme in laparoscopic appendectomy: description, validation and evaluation

Salient haptic skills trainer: initial validation of a novel simulator for training force-based laparoscopic surgical skills

Laparoscopic cholecystectomy critical view of safety (LC-CVS): a multi-national validation study of an objective, procedure-specific assessment using video-based assessment (VBA)