Abstract:<section class="article-section__content"><h3 class="article-section__sub-title section1"> Background</h3><p>Research shows that female trainees experience evaluation penalties for gender non-conforming behavior during medical training. Studies of medical education evaluations and performance scores do reflect a gender bias, though studies are of varying methodology and results have not been consistent.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Objective</h3><p>We sought to examine the differences in word use, competency themes, and length within written evaluations of internal medicine residents at scale, considering the impact of both faculty and resident gender.</p><p>We hypothesized that female internal medicine residents receive more negative feedback, and different thematic feedback than male residents.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Methods</h3><p>This study utilized a corpus of 3864 individual responses to positive and negative questions over the course of six years (2012-2018) within Yale University School of Medicine's internal medicine residency. Researchers developed a sentiment model to assess the valence of evaluation responses. We then used natural language processing (NLP) to evaluate whether female versus male residents received more positive or negative feedback and if that feedback focused on different Accreditation Council for Graduate Medical Education (ACGME) core competencies based on their gender. Evaluator-evaluatee gender dyad was analyzed to see how it impacted quantity and quality of feedback.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Results</h3><p>We found that female and male residents did not have substantively different numbers of positive or negative comments. While certain competencies were discussed more than others, gender did not seem to influence which competencies were discussed. Neither gender trainee received more written feedback, though female evaluators tended to write longer evaluations.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Conclusions</h3><p>We conclude that when examined at scale, quantitative gender differences are not as prevalent as has been seen in qualitative work. We suggest that further investigation of linguistic phenomena (such as context) is warranted to reconcile this finding with prior work.</p></section>

Differences in Standardized Letter of Evaluation (SLOE) 2.0 Scoring Between Men and Women as well as Under-represented in Medicine and Non-underrepresented in Medicine Applicants

Comparing the Standardized Letter of Evaluation (SLOE) 2.0 with SLOE for Non-residency-based EM Physicians

Home vs Away Rotation Differences in the Standardized Letters of Evaluation (SLOE) 2.0

The Standardized Letter of Evaluation: How We Perceive the Quiet Student

The Emergency Medicine Group Standardized Letter of Evaluation as a Workplace-based Assessment: The Validity Is in the Detail

Emergency Medicine Clerkship Grading Scheme, Grade, and Rank-List Distribution as Reported on Standardized Letters of Evaluation

Faculty consensus on competitiveness for the new competency-based emergency medicine standardized letter of evaluation

Dear Program Director: An Evaluation of Implicit Bias in Letters of Recommendation for Neurosurgery Residency

Gender Differences in Clinical Performance Assessment of Internal Medicine Residents: A Longitudinal Analysis of the Influence of Faculty and Trainee Gender

Decoding competitiveness: Exploring how emergency medicine faculty interpret standardized letters of evaluation

Differences in Narrative Language in Evaluations of Medical Students by Gender and Under-represented Minority Status

Evaluating the 2023 Standardized Letter of Recommendation Form in Applications to Orthopaedic Surgery Residency

Gender bias in colorectal surgery fellowship letters of recommendation

Evaluating the Impact of the Standardized Letter of Recommendation on a Successful Match in Urology Residency

Perception of Quiet Students in Emergency Medicine: An Exploration of Narratives in the Standardized Letter of Evaluation

Gender bias in resident evaluations: Natural language processing and competency evaluation

Do Gender Differences Exist in Letters of Recommendation for Reproductive Endocrinology and Infertility Fellowship?

How Prominent Is Score Inflation, Gender, and Racial Bias in Orthopaedic Surgery Residency Recommendation Letters? A Systematic Review

Gender bias in postgraduate year one pharmacy letters of recommendation

Gender Differences in Self-Description: A Linguistic Analysis of Orthopedic Surgery Residency Application Personal Statements

Public Board Score Reporting Undermines Holistic Review for Residency Selection