Abstract:Context: The benefits of programmatic assessment are well-established. Evidence from multiple assessment formats is accumulated and triangulated to inform progression committee decisions. Committees are consistently challenged to ensure consistency and fairness in programmatic deliberations. Traditional statistical and psychometric techniques are not well-suited to aggregating different assessment formats accumulated over time. Some of the strengths of programmatic assessment are also vulnerabilities viewed through this lens. While emphasis is often placed on data richness and considered input of qualified experts, committees reasonably wish for practical, defensible solutions to these challenges. Methods: We draw upon on existing literature regarding Bayesian Networks (BN), noting their utility and application in educational systems. We provide illustrative examples of how they could potentially be used in contexts that embed programmatic principles. We show a simple BN for a knowledge domain before presenting a full-scale 'proof of concept' BN to support committee decisions. We zoom in on one 'node' to demonstrate the capacity of incorporating disparate evidence throughout the network. Conclusions: Bayesian Networks offer an approach that is theoretically well-supported for programmatic assessment. They can aid committees in managing evidence accumulation, help them make inferences under conditions of uncertainty, and buttress decisions by adding a layer of defensibility to the process. They are a pragmatic tool adding value to the programmatic space by applying a complementary statistical framework. We see four major benefits of BNs in programmatic assessment: BNs allow for visual capturing of evidentiary arguments by committees during decision-making; 'recommendations' from probabilistic pathways can be used by committees to confirm their qualitative judgments; BNs can ensure precedents are maintained and consistency occurs over time; and the imperative to capture data richness is maintained without resorting to questionable methodological strategies such as adding qualitatively different things together. Further research into their feasibility and robustness in practice is warranted.

Bayes Nets in Educational Assessment: Where Do the Numbers Come From?

The potential use of Bayesian Networks to support committee decisions in programmatic assessment

Novick Meets Bayes: Improving the Assessment of Individual Students in Educational Practice and Research by Capitalizing on Assessors' Prior Beliefs

Adaptive user support in educational environments: A Bayesian Network approach

Bayesian networks applied analysis in student model modeling

Information-Theoretic Scoring Rules to Learn Additive Bayesian Network Applied to Epidemiology

An application of Bayesian inference to examine student retention and attrition in the STEM classroom

Modelling Assessment Rubrics through Bayesian Networks: a Pragmatic Approach

An Improved Parameter-Estimating Method in Bayesian Networks Applied for Cognitive Diagnosis Assessment

Using a Probabilistic Student Model to Control Problem Difficulty

Advances in Bayesian network modelling: Integration of modelling technologies

Bayesian networks and probabilistic reasoning about scientific evidence when there is a lack of data

CGBayesNets: Conditional Gaussian Bayesian Network Learning and Inference with Mixed Discrete and Continuous Data

Auto Diagnosing: An Intelligent Assessment System Based on Bayesian Networks

BNMI-DINA: A Bayesian Cognitive Diagnosis Model for Enhanced Personalized Learning

Applying the Network Item Response Model to Student Assessment Data

Dynamic Knowledge Inference Based on Bayesian Network Learning

Online Assessment of Mental Health Micromedia for College Students Incorporating Bayesian Network Algorithm

Bayesian analysis of dynamic item response models in educational testing

Intelligent tutoring systems by Bayesian networks with noisy gates

Bayesian Model Selection for High-Dimensional Ising Models, With Applications to Educational Data