Abstract:Importance: Surgical data scientists lack video data sets that depict adverse events, which may affect model generalizability and introduce bias. Hemorrhage may be particularly challenging for computer vision-based models because blood obscures the scene. Objective: To assess the utility of the Simulated Outcomes Following Carotid Artery Laceration (SOCAL)-a publicly available surgical video data set of hemorrhage complication management with instrument annotations and task outcomes-to provide benchmarks for surgical data science techniques, including computer vision instrument detection, instrument use metrics and outcome associations, and validation of a SOCAL-trained neural network using real operative video. Design, setting, and participants: For this quailty improvement study, a total of 75 surgeons with 1 to 30 years' experience (mean, 7 years) were filmed from January 1, 2017, to December 31, 2020, managing catastrophic surgical hemorrhage in a high-fidelity cadaveric training exercise at nationwide training courses. Videos were annotated from January 1 to June 30, 2021. Interventions: Surgeons received expert coaching between 2 trials. Main outcomes and measures: Hemostasis within 5 minutes (task success, dichotomous), time to hemostasis (in seconds), and blood loss (in milliliters) were recorded. Deep neural networks (DNNs) were trained to detect surgical instruments in view. Model performance was measured using mean average precision (mAP), sensitivity, and positive predictive value. Results: SOCAL contains 31 443 frames with 65 071 surgical instrument annotations from 147 trials with associated surgeon demographic characteristics, time to hemostasis, and recorded blood loss for each trial. Computer vision-based instrument detection methods using DNNs trained on SOCAL achieved a mAP of 0.67 overall and 0.91 for the most common surgical instrument (suction). Hemorrhage control challenges standard object detectors: detection of some surgical instruments remained poor (mAP, 0.25). On real intraoperative video, the model achieved a sensitivity of 0.77 and a positive predictive value of 0.96. Instrument use metrics derived from the SOCAL video were significantly associated with performance (blood loss). Conclusions and relevance: Hemorrhage control is a high-stakes adverse event that poses unique challenges for video analysis, but no data sets of hemorrhage control exist. The use of SOCAL, the first data set to depict hemorrhage control, allows the benchmarking of data science applications, including object detection, performance metric development, and identification of metrics associated with outcomes. In the future, SOCAL may be used to build and validate surgical data science models.

Perspectives on Surgical Data Science

Surgical Data Science: Enabling Next-Generation Surgery

Surgical Data Science: A Consensus Perspective

Surgical data science - from concepts toward clinical translation

Surgical data science and artificial intelligence for surgical education

SAGES consensus recommendations on surgical video data use, structure, and exploration (for research in artificial intelligence, clinical quality improvement, and surgical education)

Digital twins as a unifying framework for surgical data science: the enabling role of geometric scene understanding

Consensus for Operating Room Multimodal Data Management: Identifying Research Priorities for Data-Driven Surgery

The role of data science in healthcare advancements: applications, benefits, and future prospects

Computer Vision in the Operating Room: Opportunities and Caveats

Perspectives on Anesthesia and Perioperative Patient Safety: Past, Present, and Future

Data Infrastructure for Improvement in Perioperative Medicine: Experience at a London Tertiary Hospital

A comprehensive survey on recent deep learning-based methods applied to surgical data

Artificial Intelligence in Surgery: The Future is Now

Surgical Sabermetrics

Data-Driven Technologies as Enablers for Value Creation in the Prevention of Surgical Site Infections: a Systematic Review

Utility of the Simulated Outcomes Following Carotid Artery Laceration Video Data Set for Machine Learning Applications

Evolution of the digital operating room: the place of video technology in surgery

Impact of artificial intelligence on the training of general surgeons of the future: a scoping review of the advances and challenges

Surgical research using national databases

AI in Surgery: Navigating Trends and Managerial Implications Through Bibliometric and Text Mining Odyssey