Abstract:Background: Surgeons need to train and certify their technical skills. This is usually done with the intervention of experts who monitor and assess trainees. Nevertheless, this is a time-consuming task that is subject to variations among evaluators. In recent decades, subjectivity has been significantly reduced through 1) the introduction of standard curricula, such as the Fundamentals of Laparoscopic Surgery (FLS) program, which measures students' performance in specific exercises, and 2) rubrics, which are widely accepted in the literature and serve to provide feedback about the overall technical skills of the trainees. Although these two elements reduce subjectivity, they do not, however, eliminate the figure of the expert evaluator, and so the process remains time consuming. Objectives: The objective of this work is to automate those parts of the work of the expert evaluator that the technology can measure objectively, using sensors to collect evidence, and visualizations to provide feedback. We designed and developed 1) a cost-effective IoT (Internet of Things) learning environment for the training and assessment of surgical technical skills and 2) visualizations supported by the literature on visual learning analytics (VLA) to provide feedback about the exercises (in real time) and overall performance (at the end of the training) of the trainee. Methods: A hybrid approach was followed based on previous research for the design of the sensor based IoT learning environment. Previous studies were used as the basis for getting best practices on the tracking of surgical instruments and on the detection of the force applied to the tissue, with a focus on reducing the costs of data collection. The monitoring of the specific exercises required the design of sensors and collection mechanisms from scratch as there is little existing research on this subject. Moreover, it was necessary to design the overall architecture to collect, process, synchronize and communicate the data coming from the different sensors to provide high-level information relevant to the end user. The information to be presented was already validated by the literature and the focus was on how to visualize this information and the optimal time for its presentation to end users. The visualizations were validated with 18 VLA experts assessing the technical aspects of the visualizations and 4 medical experts assessing their functional aspects. Results: This IoT learning environment amplifies the evaluation mechanisms already validated by the literature, allowing automatic data collection. First, it uses IoT sensors to automatically correct two of the exercises defined in the FLS (peg transfer and precision cutting), providing real-time visualizations. Second it monitors the movement of the surgical instruments and the force applied to the tissues during the exercise, computing 6 of the high-level indicators used by expert evaluators in their rubrics (efficiency, economy of movement, hand tremor, depth perception, bimanual dexterity, and respect for tissue), providing feedback about the technical skills of the trainee using a radar chart with these six indicators at the end of the training (summative visualizations). Conclusions: The proposed IoT learning environment is a promising and cost-effective alternative to help in the training and assessment of surgical technical skills. The system shows the trainees' progress and presents new indicators about the correctness of each specific exercise through real-time visualizations, as well as their general technical skills through summative visualizations, aligned with the 6 more frequent indicators in standardized scales. Early results suggest that although both types of visualizations are useful, it is necessary to reduce the cognitive load of the graphs presented in real time during training. Nevertheless, an additional evaluation is needed to confirm these results.

S868 A Novel Robotic Platform for Objective Technical Skill Assessment in Colonoscopy

Using Artificial Intelligence to Gauge Competency on a Novel Laparoscopic Training System

Preliminary Design of a Robotic System for Kinaesthetic Training of Surgical Tasks

Assessing system operation skills in robotic surgery trainees

Image guided interpedicular screw placement simulation system for training and skill evaluation. Proof of concept

Toward Personalized Training and Skill Assessment in Robotic Minimally Invasive Surgery

Content and construct validation of a robotic surgery curriculum using an electromagnetic instrument tracker

Using AI and computer vision to analyze technical proficiency in robotic surgery

A novel assessment model for teaching robot-assisted living donor nephrectomy in abdominal transplant surgery fellowship

Usability, Ergonomics, and Educational Value of a Novel Telestration Tool for Surgical Coaching: Usability Study

Development and evaluation of a societal core robotic surgery accreditation curriculum for the UK

Objective Assessment of Robotic Surgical Technical Skill: A Systematic Review

Affordable Mobile-based Simulator for Robotic Surgery

Implementation and Validation of an Automated, Longitudinal Robotic Surgical Evaluation and Feedback Program at a High-volume Center and Impact on Training

Computer-assisted training system for knee arthroscopy

A Robotic Recording and Playback Platform for Training Surgeons and Learning Autonomous Behaviors Using the da Vinci Surgical System

A cost-effective IoT learning environment for the training and assessment of surgical technical skills with visual learning analytics

Development and validation of a tool for non-technical skills evaluation in robotic surgery-the ICARS system

Creation and validation of a novel low-cost dry lab for early resident training and assessment of robotic prostatectomy technical proficiency

Video-based Surgical Skills Assessment using Long term Tool Tracking

Implementation of an Online Intraoperative Assessment of Technical Performance for Surgical Trainees