Abstract:Phenomenon: Programmatic assessment and competency-based education have highlighted the need to make robust high-stakes assessment decisions on learner performance from evidence of varying types and quality. Without guidance, lengthy deliberations by decision makers and competence committees can end inconclusively with unresolved concerns. These decisional dilemmas are heightened by their potential impacts. For learners, erroneous decisions may lead to an unjustified exit from a long-desired career, or premature promotion to clinical responsibilities. For educators, there is the risk of wrongful decision-making, leading to successful appeals and mistrust. For communities, ill-prepared graduates risk the quality and safety of care. Approaches such as psychometric analyses are limited when decision-makers are faced with seemingly contradictory qualitative and quantitative evidence about the same individual. Expertise in using such evidence to make fair and defensible decisions is well established in judicial practice but is yet to be practically applied to assessment decision-making. Approach: Through interdisciplinary exchange, we investigated medical education and judicial perspectives on decision-making to explore whether principles of decision-making in law could be applied to educational assessment decision-making. Using Dialogic Inquiry, an iterative process of scholarly and mutual critique, we contrasted assessment decision making in medical education with judicial practice to identify key principles in judicial decision-making relevant to educational assessment decisions. We developed vignettes about common but problematic high-stakes decision-making scenarios to test how these principles could apply. Findings: Over 14 sessions, we identified, described, and applied four principles for fair, reasonable, and transparent assessment decision-making. These were: The person whose interests are affected has a right to know the case against them, and to be heard.Reasons for the decision should be given.Rules should be transparent and consistently applied.Like cases should be treated alike and unlike cases treated differently.Reflecting our dialogic process, we report findings by separately presenting the medical educator and judicial perspectives, followed by a synthesis describing a preferred approach to decision-making in three vignettes. Insights: Judicial principles remind educators to consider both sides of arguments, to be consistent, and to demonstrate transparency when making assessment decisions. Dialogic Inquiry is a useful approach for generating interdisciplinary insights on challenges in medical education by critiquing difference (e.g., the meaning of objectivity) and achieving synthesis where possible (e.g., fairness is not equal treatment of all cases). Our principles and exemplars provide groundwork for promoting good practice and furthering assessment research toward fairer and more robust decisions that will assist learning.

The practice of health care: wisdom as a model

Using practical wisdom to facilitate ethical decision-making: a major empirical study of phronesis in the decision narratives of doctors

Clinical reasoning performance assessment: using situated cognition theory as a conceptual framework

Rethinking the doctor-patient relationship: toward a hermeneutically-informed epistemology of medical practice

Nurses' Wisdom in Action in the Emergency Department

Professional judgement in clinical practice (part 3): A better alternative to strong evidence-based medicine

The epistemically virtuous clinician

Re-visioning clinical reasoning, or stepping out from the skull

Syncing Our Global Thinking: A Framework for Contextualized Clinical Reasoning

Industrial Conceptualization of Health Care Versus the Naturalistic Decision-Making Paradigm: Work as Imagined Versus Work as Done

Practice Rationale Care Model: The Art and Science of Clinical Reasoning, Decision Making and Judgment in the Nursing Process

Attending to Variable Interpretations of Assessment Science and Practice

Medical ethics research between theory and practice

Educators as Judges: Applying Judicial Decision-Making Principles to High-Stakes Education Assessment Decisions

The inseparability of context and clinical reasoning

Practice wisdom: the art and science of social work

Dual process models of clinical reasoning: The central role of knowledge in diagnostic expertise

Examining the Influence of Context and Professional Culture on Clinical Reasoning Through Rhetorical-Narrative Analysis

Conceptualizing Wise Management Decision-Making: A Grounded Theory Approach

Relevance of the rationalist-intuitionist debate for ethics and professionalism in medical education

Naturalistic Decision Making and the Practice of Health Care