Abstract:Phenomenon: Programmatic assessment and competency-based education have highlighted the need to make robust high-stakes assessment decisions on learner performance from evidence of varying types and quality. Without guidance, lengthy deliberations by decision makers and competence committees can end inconclusively with unresolved concerns. These decisional dilemmas are heightened by their potential impacts. For learners, erroneous decisions may lead to an unjustified exit from a long-desired career, or premature promotion to clinical responsibilities. For educators, there is the risk of wrongful decision-making, leading to successful appeals and mistrust. For communities, ill-prepared graduates risk the quality and safety of care. Approaches such as psychometric analyses are limited when decision-makers are faced with seemingly contradictory qualitative and quantitative evidence about the same individual. Expertise in using such evidence to make fair and defensible decisions is well established in judicial practice but is yet to be practically applied to assessment decision-making. Approach: Through interdisciplinary exchange, we investigated medical education and judicial perspectives on decision-making to explore whether principles of decision-making in law could be applied to educational assessment decision-making. Using Dialogic Inquiry, an iterative process of scholarly and mutual critique, we contrasted assessment decision making in medical education with judicial practice to identify key principles in judicial decision-making relevant to educational assessment decisions. We developed vignettes about common but problematic high-stakes decision-making scenarios to test how these principles could apply. Findings: Over 14 sessions, we identified, described, and applied four principles for fair, reasonable, and transparent assessment decision-making. These were: The person whose interests are affected has a right to know the case against them, and to be heard.Reasons for the decision should be given.Rules should be transparent and consistently applied.Like cases should be treated alike and unlike cases treated differently.Reflecting our dialogic process, we report findings by separately presenting the medical educator and judicial perspectives, followed by a synthesis describing a preferred approach to decision-making in three vignettes. Insights: Judicial principles remind educators to consider both sides of arguments, to be consistent, and to demonstrate transparency when making assessment decisions. Dialogic Inquiry is a useful approach for generating interdisciplinary insights on challenges in medical education by critiquing difference (e.g., the meaning of objectivity) and achieving synthesis where possible (e.g., fairness is not equal treatment of all cases). Our principles and exemplars provide groundwork for promoting good practice and furthering assessment research toward fairer and more robust decisions that will assist learning.

The Actuarial Turn in the Science of Learning Disabilities

Validity of early identification and early intervention in autism spectrum disorders: Future directions

Clinical Assessment in School Psychology: Impervious to Scientific Reform?

Attention Deficit Hyperactivity Disorder, Learning Disorders, and Other Incentivized Diagnoses — A Special Issue for Psychologists

Foreign Language Learning and Learning Disabilities: An Historical, Empirical, and Cultural Perspective for Policies and Procedures

Disability Diagnoses: Seven Sins of Clinicians

The Changing Character of Behavioral Assessment and Treatment: A Historical Introduction and Review of Functional Analysis Research

'Learning Difficulties', the Social Model of Disability and Impairment: Challenging epistemologies

FUNCTIONAL ASSESSMENT, CURRICULAR REVISION, AND SEVERE BEHAVIOR PROBLEMS

Advances in Our Understanding of Behavioral Intervention: 1980 to 2020 for Individuals Diagnosed with Autism Spectrum Disorder

Chasing Change: Repeated-Measures Analysis of Variance Is So Yesterday!

Exploring the Views, Perspectives, and Current Practices of Educational Speech-Language Pathologists and Psychologists in Canada: How Childhood Developmental Language Disorders Are Identified and Diagnosed

Multiple intelligences and underachievement: lessons from individuals with learning disabilities

Cognitive Diagnosis in Language Assessment: A Thematic Review

University students with learning disabilities advocating for change

Standardized Assessment Accommodations for Individuals with Intellectual Disability

New directions in assessment for students with disabilities

On (Cognitive) Bias in School Psychology

Disability Justice and Anti-ableism for the Pediatric Clinician

How experimental methods shaped views on human competence and rationality.

Educators as Judges: Applying Judicial Decision-Making Principles to High-Stakes Education Assessment Decisions