Narrative Performance Level Assignments at Initial Entrustment and Graduation: Integrating EPAs and Milestones to Improve Learner Assessment
Daniel J. Schumacher,Alan Schwartz,Joseph A. Zenel,Nicole Paradise Black,Keith Ponitz,Robyn Blair,Christin M. Traba,Sue Poynter,Beth King,Robert Englander,Adam Rosenberg,Dilipkumar Patel,Candace Smith-King,Meghan O’Connor,Javier Gonzalez del Rey,Suzanne Lavoie,Emily Borman-Shoap,Carol Carraccio,,
DOI: https://doi.org/10.1097/acm.0000000000003300
IF: 7.84
2020-03-17
Academic Medicine
Abstract:<span>Purpose: To determine which narrative performance level for each general pediatrics EPA reflects the minimum level clinical competency committees (CCCs) felt should be associated with graduation as well as initial entrustment and compare the narratives associated with the expected narrative performance levels (ENPLs) for each EPA with actual narrative performance levels (ANPLs) assigned to residents at initial entrustment.Method: A series of 5 narratives, corresponding to the 5 milestone performance levels, were developed for each of the 17 general pediatrics EPAs. In AY 2015–2016, the CCCs at 22 Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network member sites reported ENPLs for initial entrustment and at time of graduation. From AYs 2015–2016 to 2017–2018, programs reported ANPLs for initial entrustment decisions. ENPLs and ANPLs were compared using a logistic mixed effects model.Results: ENPLs for graduation and entrustment were most often level 3 (competent) followed by level 4 (proficient). For 8 EPAs, the ENPLs for graduation and entrustment were the same. For the remaining 9, some programs would entrust residents before graduation or graduate them before entrusting them. There were 4,266 supervision level reports for initial entrustment for which an ANPL was provided. ANPLs that were lower than the ENPLs were significantly more likely to be assigned to the medical home - well child (OR = 0.39; 95% CI: 0.26–0.57), transition to adult care (OR = 0.43; 95% CI: 0.19–0.95), behavioral or mental health (OR = 0.36; 95% CI: 0.18–0.71), make referrals (OR = 0.31; 95% CI: 0.17–0.55), lead a team (OR = 0.34; 95% CI: 0.22–0.52), and handovers (OR = 0.18; 95% CI: 0.09–0.36) EPAs.Conclusions: CCCs reported lower ENPLs for graduation than for entrustment for 5 EPAs, possibly indicating curricular gaps that milestones and EPAs could help identify.Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A832 and http://links.lww.com/ACADMED/A833.Acknowledgments: The Association of Pediatric Program Directors (APPD) Longitudinal Educational Assessment Research Network (LEARN) General Pediatrics Entrustable Professional Activities Study Group includes: Christin Traba, MD, MPH (Rutgers New Jersey Medical School); Daniel West, MD (University of California, San Francisco School of Medicine); Keith Ponitz, MD (Case Western Reserve University School of Medicine and Rainbow Babies and Children's Hospital); Adam Rosenberg, MD (University of Colorado School of Medicine and Children's Hospital Colorado); Robyn Blair, MD (Renaissance School of Medicine at Stony Brook University); Dilipkumar Patel, MD, MBA, MPH (Western Michigan University Homer Stryker M.D. School of Medicine); Nicole Paradise Black, MD, MEd (University of Florida College of Medicine); Joseph A. Zenel Jr, MD (University of South Dakota Sanford School of Medicine); Elena Griego, MD (University of Washington School of Medicine and Seattle Children's Hospital); Leah Millstein, MD (University of Maryland School of Medicine); Erin Giudice, MD (University of Maryland School of Medicine); Julie Baughn, MD (Mayo Clinic Alix School of Medicine); Su-Ting Li, MD, MPH (University of California, Davis School of Medicine); Sue Poynter, MD, MEd, (University of Cincinnati College of Medicine and Cincinnati Children's Hospital Medical Center); Javier Gonzalez del Rey, MD, MEd, (University of Cincinnati College of Medicine and Cincinnati Children's Hospital Medical Center); Daniel J. Schumacher, MD, MEd (University of Cincinnati College of Medicine and Cincinnati Children's Hospital Medical Center); Valera Hudson, MD (Medical College of Georgia); Lauren Newhall, DO (Medical College of Georgia); Teri Turner, MD, MEd, MPH (Baylor College of Medicine and Texas Children's Hospital); Hannah Famiglietti, MD (New York University School of Medicine); Kenya McNeal Trice, MD (University of North Carolina School of Medicine); Candace Smith-King, MD (Michigan State University College of Human Medicine and Helen DeVos Children's Hospital); Bruce Herman, MD (University of Utah School of Medicine); Joni Hemond, MD (University of Utah School of Medicine); Meghan O'Connor, MD (University of Utah School of Medicine); Emily Borman-Shoap, MD (University of Minnesota Medical School); Robert Englander, MD, MPH (University of Minnesota Medical School); Suzanne Lavoie, MD (Virginia Commonwealth University School of Medicine); Kimberly Gifford, MD (Geisel School of Medicine at Dartmouth University); Alan Schwartz, PhD (University of Illinois at Chicago College of Medicine and APPD LEARN); Beth King, MPP (APPD LEARN); and Carol Carraccio, MD, MA (American Board of Pediatrics).Funding/Support: This study was funded by the American Board of Pediatrics Foundation.Other disclosures: None reported.Ethical approval: The institutional review board at Cincinnati Children <p>-Abstract Truncated-</p>
education, scientific disciplines,health care sciences & services