Abstract:Importance: A high level of surgical skill is essential to prevent intraoperative problems. One important aspect of surgical education is surgical skill assessment, with pertinent feedback facilitating efficient skill acquisition by novices.Objectives: To develop a 3-dimensional (3-D) convolutional neural network (CNN) model for automatic surgical skill assessment and to evaluate the performance of the model in classification tasks by using laparoscopic colorectal surgical videos.Design, Setting, and Participants: This prognostic study used surgical videos acquired prior to 2017. In total, 650 laparoscopic colorectal surgical videos were provided for study purposes by the Japan Society for Endoscopic Surgery, and 74 were randomly extracted. Every video had highly reliable scores based on the Endoscopic Surgical Skill Qualification System (ESSQS, range 1-100, with higher scores indicating greater surgical skill) established by the society. Data were analyzed June to December 2020.Main Outcomes and Measures: From the groups with scores less than the difference between the mean and 2 SDs, within the range spanning the mean and 1 SD, and greater than the sum of the mean and 2 SDs, 17, 26, and 31 videos, respectively, were randomly extracted. In total, 1480 video clips with a length of 40 seconds each were extracted for each surgical step (medial mobilization, lateral mobilization, inferior mesenteric artery transection, and mesorectal transection) and separated into 1184 training sets and 296 test sets. Automatic surgical skill classification was performed based on spatiotemporal video analysis using the fully automated 3-D CNN model, and classification accuracies and screening accuracies for the groups with scores less than the mean minus 2 SDs and greater than the mean plus 2 SDs were calculated.Results: The mean (SD) ESSQS score of all 650 intraoperative videos was 66.2 (8.6) points and for the 74 videos used in the study, 67.6 (16.1) points. The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%). The highest accuracy was 83.8% for the inferior mesenteric artery transection. The model also screened for the group with scores less than the mean minus 2 SDs with 94.1% sensitivity and 96.5% specificity and for group with greater than the mean plus 2 SDs with 87.1% sensitivity and 86.0% specificity.Conclusions and Relevance: The results of this prognostic study showed that the proposed 3-D CNN model classified laparoscopic colorectal surgical videos with sufficient accuracy to be used for screening groups with scores greater than the mean plus 2 SDs and less than the mean minus 2 SDs. The proposed approach was fully automatic and easy to use for various types of surgery, and no special annotations or kinetics data extraction were required, indicating that this approach warrants further development for application to automatic surgical skill assessment.

Video-based automatic hand hygiene detection for operating rooms using 3D convolutional neural networks

Fully Automated Hand Hygiene Monitoring\\in Operating Room using 3D Convolutional Neural Network

A Deep Learning Based Automated Hand Hygiene Training System

A Pilot Study of Deep Learning Models for Camera based Hand Hygiene Monitoring in ICU

Hand Hygiene Assessment via Joint Step Segmentation and Key Action Scorer

Exploiting CCTV Cameras for Hand Hygiene Recognition in ICU

Video-based surgical skill assessment using 3D convolutional neural networks

Hand Hygiene Video Classification Based on Deep Learning

Real-time hand rubbing quality estimation using deep learning enhanced by separation index and feature-based confidence metric

Deep Learning Detection of Hand Motion During Microvascular Anastomosis Simulations Performed by Expert Cerebrovascular Neurosurgeons

228 Expert Hand Motion During Microvascular Anastomosis Simulation: Big Data Analysis Using Machine Learning Hand Detection

Deep learning-based monitoring technique for real-time intravenous medication bag status

Development and Validation of a 3-Dimensional Convolutional Neural Network for Automatic Surgical Skill Assessment Based on Spatiotemporal Video Analysis

Towards Vision-Based Smart Hospitals: A System for Tracking and Monitoring Hand Hygiene Compliance

Evaluating surgical skills from kinematic data using convolutional neural networks

Feature Extraction, Classification and Prediction for Hand Hygiene Gestures with KNN Algorithm

Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks

Face detection in the operating room: comparison of state-of-the-art methods and a self-supervised approach

Using hand pose estimation to automate open surgery training feedback

Innovative healthcare solutions: robust hand gesture recognition of daily life routines using 1D CNN

Handwashing Action Detection System for an Autonomous Social Robot