Abstract:Phenomenon: Programmatic assessment and competency-based education have highlighted the need to make robust high-stakes assessment decisions on learner performance from evidence of varying types and quality. Without guidance, lengthy deliberations by decision makers and competence committees can end inconclusively with unresolved concerns. These decisional dilemmas are heightened by their potential impacts. For learners, erroneous decisions may lead to an unjustified exit from a long-desired career, or premature promotion to clinical responsibilities. For educators, there is the risk of wrongful decision-making, leading to successful appeals and mistrust. For communities, ill-prepared graduates risk the quality and safety of care. Approaches such as psychometric analyses are limited when decision-makers are faced with seemingly contradictory qualitative and quantitative evidence about the same individual. Expertise in using such evidence to make fair and defensible decisions is well established in judicial practice but is yet to be practically applied to assessment decision-making. Approach: Through interdisciplinary exchange, we investigated medical education and judicial perspectives on decision-making to explore whether principles of decision-making in law could be applied to educational assessment decision-making. Using Dialogic Inquiry, an iterative process of scholarly and mutual critique, we contrasted assessment decision making in medical education with judicial practice to identify key principles in judicial decision-making relevant to educational assessment decisions. We developed vignettes about common but problematic high-stakes decision-making scenarios to test how these principles could apply. Findings: Over 14 sessions, we identified, described, and applied four principles for fair, reasonable, and transparent assessment decision-making. These were: The person whose interests are affected has a right to know the case against them, and to be heard.Reasons for the decision should be given.Rules should be transparent and consistently applied.Like cases should be treated alike and unlike cases treated differently.Reflecting our dialogic process, we report findings by separately presenting the medical educator and judicial perspectives, followed by a synthesis describing a preferred approach to decision-making in three vignettes. Insights: Judicial principles remind educators to consider both sides of arguments, to be consistent, and to demonstrate transparency when making assessment decisions. Dialogic Inquiry is a useful approach for generating interdisciplinary insights on challenges in medical education by critiquing difference (e.g., the meaning of objectivity) and achieving synthesis where possible (e.g., fairness is not equal treatment of all cases). Our principles and exemplars provide groundwork for promoting good practice and furthering assessment research toward fairer and more robust decisions that will assist learning.

Assessment, feedback and the alchemy of learning

Assessment IS learning: developing a student‐centred approach for assessment in Higher Education

The need to disentangle assessment and feedback in higher education

A model for educational feedback based on clinical communication skills strategies: beyond the "feedback sandwich"

Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice

Changing the culture of assessment: the dominance of the summative assessment paradigm

Improving the student learning experience through dialogic feed-forward assessment

Understanding the processes, practices and influences of calibration on feedback literacy in higher education marking: A qualitative study

A feedback journey: employing a constructivist approach to the development of feedback literacy among health professional learners

Ready, willing and able: a model to explain successful use of feedback

Governance models for collaborations involving assessment

Attending to Variable Interpretations of Assessment Science and Practice

‘This is not what I need’: conflicting assessment feedback beliefs in a post-secondary institution in Hong Kong

Feedback in clinical education: untying the Gordian knot

"Tell me what is 'better'!" How medical students experience feedback, through the lens of self-regulatory learning

Alternative perspectives on learning outcomes: challenges for assessment

Feelings About Feedback: The Role of Emotions in Assessment for Learning

Educators as Judges: Applying Judicial Decision-Making Principles to High-Stakes Education Assessment Decisions

Inquiring into children’s experiences of teacher feedback: reconceptualising Assessment for Learning

Teacher feedback literacy, feedback regimes and iterative change: towards enhanced value in feedback processes

From constructive to critical and everywhere in between: education leaders' decision-making related to harsh feedback from learners about their teachers