Abstract:Major vascular injury resulting in uncontrolled bleeding is a catastrophic and often fatal complication of minimally invasive surgery. At the outset of these events, surgeons do not know how much blood will be lost or whether they will successfully control the hemorrhage (achieve hemostasis). We evaluate the ability of a deep learning neural network (DNN) to predict hemostasis control ability using the first minute of surgical video and compare model performance with human experts viewing the same video. The publicly available SOCAL dataset contains 147 videos of attending and resident surgeons managing hemorrhage in a validated, high-fidelity cadaveric simulator. Videos are labeled with outcome and blood loss (mL). The first minute of 20 videos was shown to four, blinded, fellowship trained skull-base neurosurgery instructors, and to SOCALNet (a DNN trained on SOCAL videos). SOCALNet architecture included a convolutional network (ResNet) identifying spatial features and a recurrent network identifying temporal features (LSTM). Experts independently assessed surgeon skill, predicted outcome and blood loss (mL). Outcome and blood loss predictions were compared with SOCALNet. Expert inter-rater reliability was 0.95. Experts correctly predicted 14/20 trials (Sensitivity: 82%, Specificity: 55%, Positive Predictive Value (PPV): 69%, Negative Predictive Value (NPV): 71%). SOCALNet correctly predicted 17/20 trials (Sensitivity 100%, Specificity 66%, PPV 79%, NPV 100%) and correctly identified all successful attempts. Expert predictions of the highest and lowest skill surgeons and expert predictions reported with maximum confidence were more accurate. Experts systematically underestimated blood loss (mean error - 131 mL, RMSE 350 mL, R2 0.70) and fewer than half of expert predictions identified blood loss > 500 mL (47.5%, 19/40). SOCALNet had superior performance (mean error - 57 mL, RMSE 295 mL, R2 0.74) and detected most episodes of blood loss > 500 mL (80%, 8/10). In validation experiments, SOCALNet evaluation of a critical on-screen surgical maneuver and high/low-skill composite videos were concordant with expert evaluation. Using only the first minute of video, experts and SOCALNet can predict outcome and blood loss during surgical hemorrhage. Experts systematically underestimated blood loss, and SOCALNet had no false negatives. DNNs can provide accurate, meaningful assessments of surgical video. We call for the creation of datasets of surgical adverse events for quality improvement research.

228 Expert Hand Motion During Microvascular Anastomosis Simulation: Big Data Analysis Using Machine Learning Hand Detection

Deep Learning Detection of Hand Motion During Microvascular Anastomosis Simulations Performed by Expert Cerebrovascular Neurosurgeons

Mapping Surgeon's Hand/Finger Motion During Conventional Microsurgery to Enhance Intuitive Surgical Robot Teleoperation

Detection of hand motion during cadaveric mastoidectomy dissections: a technical note

Assessment of changes in vessel area during needle manipulation in microvascular anastomosis using a deep learning-based semantic segmentation algorithm: A pilot study

Using Artificial Intelligence to Gauge Competency on a Novel Laparoscopic Training System

Video-based fully automatic assessment of open surgery suturing skills

Towards Finger Motion Tracking and Analyses for Cardiac Surgery

Computer Vision for Assessing Surgical Movements in Neurosurgery

Expert surgeons and deep learning models can predict the outcome of surgical hemorrhage from 1 min of video

Using hand pose estimation to automate open surgery training feedback

Robust hand tracking for surgical telestration

Measuring hand movement for suturing skill assessment: A simulation-based study

Deep Learning with Convolutional Neural Network for Objective Skill Evaluation in Robot-assisted Surgery

Deep Learning Analysis of Surgical Video Recordings to Assess Nontechnical Skills

Analyzing the Evolution of Needle and Ultrasound Probe Manipulation Skills of Interventional Radiology Trainees With Time and Experience

Video-based Surgical Skills Assessment using Long term Tool Tracking

Laparoscopic distal gastrectomy skill evaluation from video: a new artificial intelligence-based instrument identification system

A biomechanical analysis of surgeon's gesture in a laparoscopic virtual scenario

Enhanced 2D Hand Pose Estimation for Gloved Medical Applications: A Preliminary Model

Objective and Automated Quantification of Instrument Handling for Open Surgical Suturing Skill Assessment: A Simulation-Based Study