Abstract:Video-based assessment (VBA) is gaining traction as an adjunctive, and potentially alternative, method to more traditional assessments of surgical technical skills during residency training and may someday play a role in professional certification. 1 In this issue of JAMA Network Open , Ketel et al 2 report the use of a previously developed, video-based, minimally invasive esophagectomy (MIE) competency assessment tool (CAT) 3 to examine the association between the intraoperative technical quality of how MIE is performed at 15 Dutch hospitals and postoperative complications. Similar to the initial seminal work reporting an association of technical quality with better patient outcomes, the authors used masked reviewers with expertise in MIE and found that hospitals with better overall technical quality in the performance of the operation had lower intraoperative and severe postoperative complication rates and better performance during the anastomosis phase was associated with lower leak rates. 2 This important work supports the concept that superior technical skill is associated with better patient outcomes. 4 ,5 It also builds on prior work by suggesting that expertise within the team (rather than just the individual) may be similarly critical. The association between better performance during the creation of anastomosis phase and lower postoperative leak rates is particularly compelling because it suggests that experts can accurately identify technical quality during the most critical steps of a procedure. However, as with anything, the devil is in the details. Therefore, we must consider the nuances of these data to fully understand their meaning and how they might eventually be operationalized. For example, hospitals that performed in the highest quartile of MIE-CAT score had a significantly lower rate of severe postoperative complications compared with those in the lowest quartile (18.7% in the highest quartile vs 39.2% in the lowest quartile). We should exercise caution, however, in interpreting this association because many postoperative complications may be affected by numerous other relevant patient-, process-, and hospital-level factors above and beyond the technical quality of the intraoperative care delivered. Another important nuance is the stratification of performance into quartiles for the purposes of this analysis. While there is certainly precedent for using this type of stratification and it is often necessary for presenting data in a cohesive and interpretable manner, our natural tendency is to equate lower-ranked performance with worse outcomes. However, this type of oversimplification can be problematic because technical ability spans a range (rather than a static rank), and increments within that range may not necessarily translate into clinically meaningful differences. Furthermore, performance may be associated differently with different outcomes, some of which may or may not be expected to be directly associated with better performance. For instance, in this study, R0 resection, lymph node yield, and 30-day mortality were not clearly associated with hospital MIE-CAT performance. While intraoperative technical quality may not necessarily directly affect postoperative mortality, it might have been reasonable to expect an association with R0 resection and lymph node yield. Because they weren't, this finding begs the question of why intraoperative and postoperative complications were associated while these other outcomes were not. It may simply be that these findings are a reminder that the goal is not, and should not be, numeric performance alone but, rather, clinically relevant competence. Because skill level represents a spectrum and a variety of skill levels may achieve the same desired result, as VBA becomes more common and takes on a larger role in evaluating surgical competence, it will be imperative to continue to anchor technical skill (as measured with these types of tools) to clinically relevant patient outcomes. A critical challenge highlighted by this study is the relatively labor-intensive nature of creating footage and completing assessments as part of VBA. That the authors cite an unfeasible workload as the main reason they could only use 2 representative videos to create a hospital-level assessment speaks directly to this concern. While the smaller team sizes at the Dutch hospitals included in this study may have facilitated a limited number of videos providing a generalizable representation of each hospital's performance, we cannot know what we haven't seen. With only 30 total videos assessed from the 15 hospitals that treated the 970 patients included in the study, potential explanations for these findings may be that (1) the authors have identified a tool that generalizes perfectly (with minimal data) to all the surgical team members and patients underg -Abstract Truncated-

A New Procedure-Based Assessment of Operative Skills in Gastric Bypass Surgery, Evaluated by Video Fragment Rating

Effect of video-based self-reflection on intraoperative skills: A pilot randomized controlled trial

Hierarchical task analysis of endoscopic sleeve gastroplasty

Surgical quality assessment of critical view of safety in 283 laparoscopic cholecystectomy videos by surgical residents and surgeons

Correlation between operative time and crowd-sourced skills assessment for robotic bariatric surgery

Video-Based Assessment of Surgical Quality

Development and Evaluation of a Quality Assessment Tool for Laparoscopic Sleeve Gastrectomy Videos: A Review and Comparison of Academic and Online Video Resources

Comparison of gaze behaviour of trainee and experienced surgeons during laparoscopic gastric bypass

Expanded Access to Video-Based Laparoscopic Skills Assessments: Ease, Reliability, and Accuracy

Global assessment of surgical skills (GASS): validation of a new instrument to measure global technical safety in surgical procedures

A Systematic Review to Summarise and Appraise the Reporting of Surgical Innovation: a Case Study in Robotic Roux-en-Y Gastric Bypass

Laparoscopic cholecystectomy critical view of safety (LC-CVS): a multi-national validation study of an objective, procedure-specific assessment using video-based assessment (VBA)

Development and preliminary validation of a new task-based objective procedure-specific assessment of inguinal hernia repair procedural safety

A multi-national, video-based qualitative study to refine training guidelines for assigning an "unsafe" score in laparoscopic cholecystectomy critical view of safety

Design and validation of an assessment tool for open surgical procedures

Video-based tools for surgical quality assessment of technical skills in laparoscopic procedures: a systematic review

ROBOTIC VERSUS LAPAROSCOPIC ROUX-EN-Y-GASTRIC BYPASS: A RETROSPECTIVE STUDY IN A SINGLE CENTER

Objective performance indicators versus GEARS: an opportunity for more accurate assessment of surgical skill

Global benchmarks in primary robotic bariatric surgery redefine quality standards for Roux-en-Y gastric bypass and sleeve gastrectomy

The Impact of Operative Video Review and Annotation on Training in Endoscopic Pituitary Surgery: A Preclinical Randomised Controlled Study

Endoscopic versus laparoscopic bariatric procedures: A computational biomechanical study through a patient-specific approach