Abstract:Introduction: Entrustable Professional Activities (EPAs) provide a framework for competency-based assessment in surgery. EPA descriptions include observable behaviors by trainees at progressive levels of autonomy. The American Board of Surgery (ABS) required all General Surgery (GS) residency programs to implement assessment of 18 EPAs at the beginning of academic year 2023-2024. Microassessments provide formative self-reflection by the resident and feedback by faculty upon completion of the EPA. These frequent assessments culminate in a resident performance profile utilized by the trainee for formative growth and the clinical competency committee for summative feedback. Assessor free text comments are an opportunity to provide meaningful, constructive feedback to residents. Our aim was to analyze comments provided by faculty to residents in terms of their alignment with EPA descriptors and provision of actionable feedback. Methods: A total of 540 GS EPA assessments for inguinal hernia, gallbladder disease, appendicitis, trauma, and surgical consultation were evaluated from 6/2021-12/2022. We assessed free text EPA comments from faculty compared to EPA behavior descriptions for alignment with the selected EPA level of entrustment. The comments were judged on a binary scale of "Align" vs "Not Align" by two independent evaluators, with a third evaluator to address discordance. Comments were then evaluated for resident behavioral descriptions, suggestions for improvement, and positive or negative feedback. Results: Approximately 77 % of EPA microassessments had alignment between level of autonomy and free text feedback. A common example of feedback discordant with level of autonomy was rating a trainee at an intraoperative level 4 (independent practice) with comments such as "required some guidance with retrocecal case and upsizing port." Based on behavior descriptions this would be a level 3 (indirect supervision). Approximately 88 % of feedback contained positive comments with minimal negative feedback (e.g., "this did not go well."). Actionable feedback including "work on optimization of retracting hand" or "continue to work clamp/tie technique and square off each knot" was present in 28.3 % of feedback. Conclusions: The majority of faculty provide feedback that is aligned with the behavioral anchors of the EPAs assessed, but frequently did not provide actionable feedback to the resident regarding how to advance to the next level of entrustment. EPA entrustment behaviors provide a framework for the development of practice-ready behaviors, and if assessors anchor their feedback in the behaviors for a given entrustment level and project how a resident could proceed to the next level, they can provide a clear trajectory for skill development. Faculty development should focus on improving the frequency of actionable free text feedback, outlining how residents can advance in the future.

Evaluating the Quality of Written Feedback Within Entrustable Professional Activities in an Internal Medicine Cohort

Evaluating Feedback Comments in Entrustable Professional Activities: A Cross-Sectional Study

Evaluating the Quality of Narrative Feedback for Entrustable Professional Activities in a Surgery Residency Program

How well do faculty do in providing general surgery EPA feedback?

Comparing Entrustable Professional Activity Scores Given by Faculty Physicians and Senior Trainees to First-Year Residents

What Faculty Want: Academic and Community Emergency Physicians’ Perceptions of Learner Feedback

Feedback Culture Perceived by Trainees in an Academic Institution: A Mixed Methods Study

EQual, a Novel Rubric to Evaluate Entrustable Professional Activities for Quality and Structure

Evaluating a medication history-taking entrustable professional activity and its assessment tool - Survey of a statewide public hospital pharmacy service

Domains Influencing Faculty Decisions on the Level of Supervision Required for Anesthesiology EPAs with Analysis of Feedback Comments

A Study to Analyze Narrative Feedback Record of an Emergency Department

Motivations for Entrustable Professional Activity Assessment: Gaps Between Curriculum Theory and Resident Reality

Entrustable Professional Activities for Chinese Standardized Residency Training in Pediatric Intensive Care Medicine

What can we expect from medical graduates? Empirical survey on the performance of Core EPAs in the first days of residency

Redesigned Entrustable Professional Activity (EPA) Assessments Reduce Grade Inflation in the Experiential Setting

From Their Eyes: What Constitutes Quality Formative Written Feedback for Neurosurgery Residents

Making assessment a team sport: a qualitative study of facilitated group feedback in internal medicine residency

Implementation and faculty perception of outpatient medical student workplace-based assessments

Addressing practical challenges with entrustable professional activities in ambulatory care experiential education

An Analysis of Workplace-Based Assessments for Core Entrustable Professional Activities for Entering Residency: Does Type of Clinical Assessor Influence Level of Supervision Ratings?

Automated near-real-time clinical performance feedback for anesthesiology residents: one piece of the milestones puzzle