Abstract:Background: It is well understood that surgical skills largely define patient outcomes both in Minimally Invasive Surgery (MIS) and Robot-Assisted MIS (RAMIS). Non-technical surgical skills, including stress and distraction resilience, decision-making and situation awareness also contribute significantly. Autonomous, technologically supported objective skill assessment can be efficient tools to improve patient outcomes without the need to involve expert surgeon reviewers. However, autonomous non-technical skill assessments are unstandardized and open for more research. Recently, Surgical Data Science (SDS) has become able to improve the quality of interventional healthcare with big data and data processing techniques (capture, organization, analysis and modeling of data). SDS techniques can also help to achieve autonomous non-technical surgical skill assessments. Methods: An MIS training experiment is introduced to autonomously assess non-technical skills and to analyse the workload based on sensory data (video image and force) and a self-rating questionnaire (SURG-TLX). A sensorized surgical skill training phantom and adjacent training workflow were designed to simulate a complicated Laparoscopic Cholecystectomy task; the dissection of the cholecyst’s peritonial layer and the safe clip application on the cystic artery in an uncomfortable environment. A total of 20 training sessions were recorded from 7 subjects (3 non-medicals, 2 residents, 1 expert surgeon and 1 expert MIS surgeon). Workload and learning curves were studied via SURG-TLX. For autonomous non-technical skill assessment, video image data with tracked instruments based on Channel and Spatial Reliability Tracker (CSRT) and force data were utilized. An autonomous time series classification was achieved by a Fully Convolutional Neural Network (FCN), where the class labels were provided by SURG-TLX. Results: With unpaired t-tests, significant differences were found between the two groups (medical professionals and control) in certain workload components (mental demands, physical demands, and situational stress, p<0.0001, 95% confidence interval, p<0.05 for task complexity). With paired t-tests, the learning curves of the trials were also studied; the task complexity resulted in a significant difference between the first and the second trials. Autonomous non-technical skill classification was based on the FCN by applying the tool trajectories and force data as input. This resulted in a high accuracy (85%) on temporal demands classification based on the z component of the used forces and 75% accuracy for classifying mental demands/situational stress with the x component of the used forces validated with Leave One Out Cross-Validation. Conclusions: Non-technical skills and workload components can be classified autonomously based on measured training data. SDS can be effective via automated non-technical skill assessment.

Surgical Data Science: Enabling Next-Generation Surgery

Surgical Data Science: A Consensus Perspective

Surgical data science - from concepts toward clinical translation

Perspectives on Surgical Data Science

Surgical data science and artificial intelligence for surgical education

SAGES consensus recommendations on surgical video data use, structure, and exploration (for research in artificial intelligence, clinical quality improvement, and surgical education)

Digital twins as a unifying framework for surgical data science: the enabling role of geometric scene understanding

Consensus for Operating Room Multimodal Data Management: Identifying Research Priorities for Data-Driven Surgery

Artificial Intelligence in Surgery: The Future is Now

A comprehensive survey on recent deep learning-based methods applied to surgical data

Surgical Sabermetrics

Role of Artificial Intelligence in Surgical Decision-Making: A Comprehensive Review

Surgomics: personalized prediction of morbidity, mortality and long-term outcome in surgery using machine learning on multimodal data

Computer Vision in the Operating Room: Opportunities and Caveats

Surgeons' perspectives on artificial intelligence to support clinical decision-making in trauma and emergency contexts: results from an international survey

Next in Surgical Data Science: Autonomous Non-Technical Skill Assessment in Minimally Invasive Surgery Training

Artificial intelligence for improving intraoperative surgical care

Artificial intelligence in surgery

Utility of the Simulated Outcomes Following Carotid Artery Laceration Video Data Set for Machine Learning Applications

The role of big data management, data registries, and machine learning algorithms for optimizing safe definitive surgery in trauma: a review

Artificial Intelligence in Surgery: Neural Networks and Deep Learning