Abstract:Problem: Development of a novel, competency-based program of assessment requires creation of a plan to measure the processes that enable successful implementation. The principles of implementation science outline the importance of considering key drivers that support and sustain transformative change within an educational program. The introduction of Entrustable Professional Activities (EPAs) as a framework for assessment has underscored the need to create a structured plan to prepare assessors to engage in a new paradigm of assessment. Although approaches to rater training for workplace-based assessments have been described, specific strategies to prepare assessors to apply standards related to the level of supervision a student needs have not been documented. Intervention: We describe our systematic approach to prepare assessors, faculty and postgraduate trainees, to complete EPA assessments for medical students during the clerkship phase of our curriculum. This institution-wide program is designed to build assessors' skills in direct observation of learners during authentic patient encounters. Assessors apply new knowledge and practice skills in using established performance expectations to determine the level of supervision a learner needs to perform clinical tasks. Assessors also learn to provide feedback and narrative comments to coach students and promote their ongoing clinical development. Data visualizations for assessors facilitate reinforcement of the tenets learned during training. Collaborative learning and peer feedback during faculty development sessions promote the formation of a community of practice among assessors. Context: Faculty development for assessors was implemented in advance of implementation of the EPA program. Assessors in the program include residents/fellows who work closely with students, faculty with discipline-specific expertise and a group of experienced clinicians who were selected to serve as experts in competency-based EPA assessments, the Master Assessors. Training focused on creating a shared understanding about the application of criteria used to evaluate student performance. EPA assessments based on the AAMC's Core Entrustable Professional Activities for Entering Residency, were completed in nine core clerkships. EPA assessments included a supervision rating based on a modified scale for use in undergraduate medical education. Impact: Data from EPA assessments completed during the first year of the program were analyzed to evaluate the effectiveness of the faculty development activities implemented to prepare assessors to consistently apply standards for assessment. A systematic approach to training and attention to critical drivers that enabled institution-wide implementation, led to consistency in the supervision rating for students' first EPA assessment completed by any type of assessor, ratings by assessors done within a specific clinical context, and ratings assigned by a group of specific assessors across clinical settings. Lessons learned: A systematic approach to faculty development with a willingness to be flexible and reach potential participants using existing infrastructure, can facilitate assessors' engagement in a new culture of assessment. Interaction among participants during training sessions not only promotes learning but also contributes to community building. A leadership group responsible to oversee faculty development can ensure that the needs of stakeholders are addressed and that a change in assessment culture is sustained.

Workplace-Based Entrustment Scales for the Core EPAs: A Multisite Comparison of Validity Evidence for Two Proposed Instruments Using Structured Vignettes and Trained Raters

A Question of Scale? Generalizability of the Ottawa and Chen Scales to Render Entrustment Decisions for the Core EPAs in the Workplace

Validity Evidence for Assessing Entrustable Professional Activities During Undergraduate Medical Education

WBAs in UME—How Many Are Needed? A Reliability Analysis of 5 AAMC Core EPAs Implemented in the Internal Medicine Clerkship

How well do workplace-based assessments support summative entrustment decisions? A multi-institutional generalisability study

Exploration of validity evidence for core residency entrustable professional activities in Chinese pediatric residency

An Analysis of Workplace-Based Assessments for Core Entrustable Professional Activities for Entering Residency: Does Type of Clinical Assessor Influence Level of Supervision Ratings?

Introducing an assessment tool based on a full set of end-of-training EPAs to capture the workplace performance of final-year medical students

"A whole other competence story": exploring faculty perspectives on the process of workplace-based assessment of entrustable professional activities

Validity of Entrustable Professional Activities in a National Sample of General Surgery Residency Programs

EQual, a Novel Rubric to Evaluate Entrustable Professional Activities for Quality and Structure

Framing our Expectations: Variability in Entrustable Professional Activity Assessments

Implementing Systematic Faculty Development to Support an EPA-Based Program of Assessment: Strategies, Outcomes, and Lessons Learned

Feasibility, and validity aspects of Entrustable Professional Activity (EPA)-based assessment in general practice training

Entrustment decisions: Implications for curriculum development and assessment

Clinical competency committee perceptions of entrustable professional activities and their value in assessing fellows: A qualitative study of pediatric subspecialty program directors

Motivations for Entrustable Professional Activity Assessment: Gaps Between Curriculum Theory and Resident Reality

Development and Pilot Testing of Entrustable Professional Activities for US Anesthesiology Residency Training

Finding a Path to Entrustment in Undergraduate Medical Education: A Progress Report From the AAMC Core Entrustable Professional Activities for Entering Residency Entrustment Concept Group

Fidelity to Best Practices in EPA Implementation: Outcomes Supporting Use of the Core Components Framework From the University of Virginia Entrustable Professional Activity Program

Constructing Approaches to Entrustable Professional Activity Development that Deliver Valid Descriptions of Professional Practice