Measuring the impact of simulation debriefing on the practices of interprofessional trauma teams using natural language processing

Alexandra A Rosser,Yazeed M Qadadha,Ryan J Thompson,Hee Soo Jung,Sarah Jung
DOI: https://doi.org/10.1016/j.amjsurg.2022.09.018
Abstract:Background: Natural language processing (NLP) may be a tool for automating trauma teamwork assessment in simulated scenarios. Methods: Using the Trauma Nontechnical Skills Assessment (T-NOTECHS), raters assessed video recordings of trauma teams in simulated pre-debrief (Sim1) and post-debrief (Sim2) trauma resuscitations. We developed codes through directed content analysis and created algorithms capturing teamwork-related discourse through NLP. Using a within subjects pre-post design (n = 150), we compared changes in teams' Sim1 versus Sim2 T-NOTECHS scores and automatically coded discourse to identify which NLP algorithms could identify skills assessed by the T-NOTECHS. Results: Automatically coded behaviors revealed significant post-debrief increases in teams' simulation discourse: Verbalizing Findings, Acknowledging Communication, Directed Communication, Directing Assessment and Role Assignment, and Leader as Hub for Information. Conclusions: Our results suggest NLP can capture changes in trauma team discourse. These findings have implications for the expedition of team assessment and innovations in real-time feedback when paired with speech-to-text technology.
What problem does this paper attempt to address?