Abstract:Context: The benefits of programmatic assessment are well-established. Evidence from multiple assessment formats is accumulated and triangulated to inform progression committee decisions. Committees are consistently challenged to ensure consistency and fairness in programmatic deliberations. Traditional statistical and psychometric techniques are not well-suited to aggregating different assessment formats accumulated over time. Some of the strengths of programmatic assessment are also vulnerabilities viewed through this lens. While emphasis is often placed on data richness and considered input of qualified experts, committees reasonably wish for practical, defensible solutions to these challenges. Methods: We draw upon on existing literature regarding Bayesian Networks (BN), noting their utility and application in educational systems. We provide illustrative examples of how they could potentially be used in contexts that embed programmatic principles. We show a simple BN for a knowledge domain before presenting a full-scale 'proof of concept' BN to support committee decisions. We zoom in on one 'node' to demonstrate the capacity of incorporating disparate evidence throughout the network. Conclusions: Bayesian Networks offer an approach that is theoretically well-supported for programmatic assessment. They can aid committees in managing evidence accumulation, help them make inferences under conditions of uncertainty, and buttress decisions by adding a layer of defensibility to the process. They are a pragmatic tool adding value to the programmatic space by applying a complementary statistical framework. We see four major benefits of BNs in programmatic assessment: BNs allow for visual capturing of evidentiary arguments by committees during decision-making; 'recommendations' from probabilistic pathways can be used by committees to confirm their qualitative judgments; BNs can ensure precedents are maintained and consistency occurs over time; and the imperative to capture data richness is maintained without resorting to questionable methodological strategies such as adding qualitatively different things together. Further research into their feasibility and robustness in practice is warranted.

Why do we need to employ Bayesian statistics and how can we employ it in studies of moral education?: With practical guidelines to use JASP for educators and researchers

Bayesian statistics in educational research: a look at the current state of affairs

Bayesian inference for psychology. Part II: Example applications with JASP

An introduction to Bayesian statistics in health psychology

The Bayes Factor and Its Implementation in JASP: A Practical Primer

Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications

Combine Statistical Thinking With Open Scientific Practice: A Protocol of a Bayesian Research Project

Bayesian Tools of the Trade for Developmental Psychologists: A Quick-Start Guide Using JASP

Bayesian alternatives to null hypothesis significance testing in biomedical research: a non-technical introduction to Bayesian inference with JASP

Bayesian Data Analysis: A Fresh Approach to Power Issues and Null Hypothesis Interpretation

Beyond 'statistical significance': A nontechnical primer of Bayesian statistics and Bayes factors for health researchers

Bayesian Analysis for Social Science Research

The potential use of Bayesian Networks to support committee decisions in programmatic assessment

Bayesian Repeated-Measures Analysis of Variance: An Updated Methodology Implemented in JASP

There is Nothing Magical about Bayesian Statistics: An Introduction to Epistemic Probabilities in Data Analysis for Psychology Starters

Bayesian Computing in the Undergraduate Statistics Curriculum

Bayesian data analysis.

Bayesian Models and Methods in Public Policy and Government Settings

Toward a principled Bayesian workflow in cognitive science.

An application of Bayesian inference to examine student retention and attrition in the STEM classroom

Bayesian Statistics in Software Engineering: Practical Guide and Case Studies