Abstract:Background: It is well understood that surgical skills largely define patient outcomes both in Minimally Invasive Surgery (MIS) and Robot-Assisted MIS (RAMIS). Non-technical surgical skills, including stress and distraction resilience, decision-making and situation awareness also contribute significantly. Autonomous, technologically supported objective skill assessment can be efficient tools to improve patient outcomes without the need to involve expert surgeon reviewers. However, autonomous non-technical skill assessments are unstandardized and open for more research. Recently, Surgical Data Science (SDS) has become able to improve the quality of interventional healthcare with big data and data processing techniques (capture, organization, analysis and modeling of data). SDS techniques can also help to achieve autonomous non-technical surgical skill assessments. Methods: An MIS training experiment is introduced to autonomously assess non-technical skills and to analyse the workload based on sensory data (video image and force) and a self-rating questionnaire (SURG-TLX). A sensorized surgical skill training phantom and adjacent training workflow were designed to simulate a complicated Laparoscopic Cholecystectomy task; the dissection of the cholecyst’s peritonial layer and the safe clip application on the cystic artery in an uncomfortable environment. A total of 20 training sessions were recorded from 7 subjects (3 non-medicals, 2 residents, 1 expert surgeon and 1 expert MIS surgeon). Workload and learning curves were studied via SURG-TLX. For autonomous non-technical skill assessment, video image data with tracked instruments based on Channel and Spatial Reliability Tracker (CSRT) and force data were utilized. An autonomous time series classification was achieved by a Fully Convolutional Neural Network (FCN), where the class labels were provided by SURG-TLX. Results: With unpaired t-tests, significant differences were found between the two groups (medical professionals and control) in certain workload components (mental demands, physical demands, and situational stress, p<0.0001, 95% confidence interval, p<0.05 for task complexity). With paired t-tests, the learning curves of the trials were also studied; the task complexity resulted in a significant difference between the first and the second trials. Autonomous non-technical skill classification was based on the FCN by applying the tool trajectories and force data as input. This resulted in a high accuracy (85%) on temporal demands classification based on the z component of the used forces and 75% accuracy for classifying mental demands/situational stress with the x component of the used forces validated with Leave One Out Cross-Validation. Conclusions: Non-technical skills and workload components can be classified autonomously based on measured training data. SDS can be effective via automated non-technical skill assessment.

Deep Learning to Automate Technical Skills Assessment in Robotic Surgery

Deep Learning with Convolutional Neural Network for Objective Skill Evaluation in Robot-assisted Surgery

Deep learning prediction of error and skill in robotic prostatectomy suturing

Using Artificial Intelligence to Gauge Competency on a Novel Laparoscopic Training System

SATR-DL: Improving Surgical Skill Assessment and Task Recognition in Robot-assisted Surgery with Deep Neural Networks

Deep neural network architecture for automated soft surgical skills evaluation using objective structured assessment of technical skills criteria

Assessing system operation skills in robotic surgery trainees

Video-based skill acquisition assessment in laparoscopic surgery using deep learning

Deep Learning Analysis of Surgical Video Recordings to Assess Nontechnical Skills

Artificial intelligence for improving intraoperative surgical care

Video-based Formative and Summative Assessment of Surgical Tasks using Deep Learning

Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks

Objective Assessment of Robotic Surgical Technical Skill: A Systematic Review

Using AI and computer vision to analyze technical proficiency in robotic surgery

Quantification of Robotic Surgeries with Vision-Based Deep Learning

An Automated Skill Assessment Framework Based on Visual Motion Signals and a Deep Neural Network in Robot-Assisted Minimally Invasive Surgery

Deep Reinforcement Learning in Surgical Robotics: Enhancing the Automation Level

Uncertainty-Aware Self-Supervised Learning for Cross-Domain Technical Skill Assessment in Robot-Assisted Surgery

Next in Surgical Data Science: Autonomous Non-Technical Skill Assessment in Minimally Invasive Surgery Training

Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions

Toward Personalized Training and Skill Assessment in Robotic Minimally Invasive Surgery