Abstract:Context: The benefits of programmatic assessment are well-established. Evidence from multiple assessment formats is accumulated and triangulated to inform progression committee decisions. Committees are consistently challenged to ensure consistency and fairness in programmatic deliberations. Traditional statistical and psychometric techniques are not well-suited to aggregating different assessment formats accumulated over time. Some of the strengths of programmatic assessment are also vulnerabilities viewed through this lens. While emphasis is often placed on data richness and considered input of qualified experts, committees reasonably wish for practical, defensible solutions to these challenges. Methods: We draw upon on existing literature regarding Bayesian Networks (BN), noting their utility and application in educational systems. We provide illustrative examples of how they could potentially be used in contexts that embed programmatic principles. We show a simple BN for a knowledge domain before presenting a full-scale 'proof of concept' BN to support committee decisions. We zoom in on one 'node' to demonstrate the capacity of incorporating disparate evidence throughout the network. Conclusions: Bayesian Networks offer an approach that is theoretically well-supported for programmatic assessment. They can aid committees in managing evidence accumulation, help them make inferences under conditions of uncertainty, and buttress decisions by adding a layer of defensibility to the process. They are a pragmatic tool adding value to the programmatic space by applying a complementary statistical framework. We see four major benefits of BNs in programmatic assessment: BNs allow for visual capturing of evidentiary arguments by committees during decision-making; 'recommendations' from probabilistic pathways can be used by committees to confirm their qualitative judgments; BNs can ensure precedents are maintained and consistency occurs over time; and the imperative to capture data richness is maintained without resorting to questionable methodological strategies such as adding qualitatively different things together. Further research into their feasibility and robustness in practice is warranted.

Novick Meets Bayes: Improving the Assessment of Individual Students in Educational Practice and Research by Capitalizing on Assessors' Prior Beliefs

Student Modeling from Conversational Test Data: A Bayesian Approach Without Priors

Bayes Nets in Educational Assessment: Where Do the Numbers Come From?

Bayesian student modeling, user interfaces and feedback: A sensitivity analysis

The potential use of Bayesian Networks to support committee decisions in programmatic assessment

Assessing Preknowledge Cheating via Innovative Measures: A Multiple-Group Analysis of Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts

Computer Adaptive Testing Using Upper-Confidence Bound Algorithm for Formative Assessment

Why do we need to employ Bayesian statistics and how can we employ it in studies of moral education?: With practical guidelines to use JASP for educators and researchers

The Bayes Factor and Its Implementation in JASP: A Practical Primer

Learning meets Assessment: On the relation between Item Response Theory and Bayesian Knowledge Tracing

An application of Bayesian inference to examine student retention and attrition in the STEM classroom

Dynamic Bayesian Network Modeling of Game-Based Diagnostic Assessments

Estimating Test-Retest Reliability in the Presence of Self-Selection Bias and Learning/Practice Effects

A Bayesian Active Learning Approach to Comparative Judgement

Shaky Student Growth? A Comparison of Robust Bayesian Learning Progress Estimation Methods

Detecting Preknowledge Cheating via Innovative Measures: A Mixture Hierarchical Model for Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts

Predicting Learning: Understanding the Role of Executive Functions in Children's Belief Revision Using Bayesian Models

Inferring constructs of effective teaching from classroom observations: An application of Bayesian exploratory factor analysis without restrictions

Adaptive user support in educational environments: A Bayesian Network approach

On the Development of a Computer-Based Tool for Formative Student Assessment: Epistemological, Methodological, and Practical Issues

Bayesian Tools of the Trade for Developmental Psychologists: A Quick-Start Guide Using JASP