Board #322 - Technology Innovation Development and Preliminary Validation of a Novel Ventriculostomy Simulator (Submission #8635)
Deborah Rooney,Peng-Siang Liao,Oren Sagher,Luis Savastano,Albert Shih,Francesca Stephenson,Bruce Tai
DOI: https://doi.org/10.1097/01.sih.0000459423.93549.f4
2014-12-01
Abstract:Introduction A team of biomedical engineers, neurosurgeons, and a medical educator, created and evaluated preliminary validity evidence of a high-fidelity simulator used to train ventriculostomy skills. Description A scale reproduction of an adult skull and relevant anatomy was created. Twelve neurosurgery residents, one fellow, and five attendings (n=17) from three academic medical centers performed the simulated creation of a ventricolostomy. Participants rated the simulator using a 37-item survey across five domains. Quality of the simulator was scored from “Not at all realistic” (1) to “Highly realistic” (4). Self-reported ability to perform relevant tasks was rated from “Too difficult to perform (0) to “Too easy to perform” (4). Validity evidence relevant to test content was evaluated using a Rasch model, while evidence relevant to internal structure (inter-item reliability, and inter-rater agreement) was evaluated using traditional methods. Conclusion Analyses indicated attendings had statistically higher ratings (M=3.4/4.0) than fellows (M=3.3), and residents (M=3.0), p=.02. Domain means were 3.9 (Value), 3.5 (Physical attributes), 3.4 (Realism of experience), 3.3 (Relevance), and 2.9 (Ability). Inter-item consistency across domains were estimated to be moderate-high (.65, .92), and inter-rater agreement regarding simulator characteristics was high [ICC(2,k)= .90]. Rating differences were found across institutions, p=.001, and are reviewed. The observed global rating (2.5) indicated the simulator can be considered for teaching ventriculostomies, but could be improved slightly. Ratings indicated the simulator was valuable as a learning tool, but could be improved with minor modifications. The most commonly-suggested improvement was the addition of ears as an added anatomical landmark. References 1. Haji FA, Dubrowski A, Drake J, de Ribaupierre S. Needs assessment for simulation training in neuroendoscopy: a Canadian national survey. J Neurosurg. 2013 Feb;118(2):250-7. doi: 10.3171/2012.10.JNS12767. Epub 2012 Dec 7 2. Schirmer CM, Elder JB, Roitberg B, Lobel DA. Virtual reality-based simulation training for ventriculostomy: an evidence-based approach. Neurosurgery. 2013 Oct;73 Suppl 1:66-73. doi: 10.1227/NEU.0000000000000074. 3. Choudhury N, Gélinas-Phaneuf N, Delorme S, Del Maestro R. Fundamentals of neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg. 2013 Nov;80(5):e9-19. doi: 10.1016/j.wneu.2012.08.022. Epub 2012 Nov 23. 4. Korndorffer JR Jr, Kasten SJ, Downing SM. A call for the utilization of consensus standards in the surgical education literature. Am J Surg. 2010 Jan;199(1):99-104. doi: 10.1016/j.amjsurg.2009.08.018 5. Cook DA, Brydges R, Zendejas B, Hamstra SJ, Hatala R. Technology-enhanced simulation to assess health professionals: A systematic review of validity evidence, research methods, and reporting quality. Acad Med. 2013 Jun;88(6):872-83. doi: 10.1097/ACM.0b013e31828ffdcf. 6. Cook DA, Zendejas B, Hamstra SJ, Hatala R, Brydges R. What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment. Adv Health Sci Educ Theory Pract. 2014 May;19(2):233-50. doi: 10.1007/s10459-013-9458-4. Epub 2013 May 2. 7. Standards for Educational and Psychological Testing, 1999, American Educational Research Association, American Psychological Association and National Council on Measurement in Education: American Educational Research. 8. Rasch G. (1960/1980). Probabilistic models for some intelligence and attainment tests. (Copenhagen, Danish Institute for Educational Research), expanded edition (1980) with foreword and afterword by BD Wright. Chicago, IL: The University of Chicago Press. 9. Hamilton JM, et al., Toward effective pediatric minimally invasive surgical simulation. J Pediatr Surg, 2011. 46(1): p. 138-44. Disclosures None
health care sciences & services