Abstract:Context: The benefits of programmatic assessment are well-established. Evidence from multiple assessment formats is accumulated and triangulated to inform progression committee decisions. Committees are consistently challenged to ensure consistency and fairness in programmatic deliberations. Traditional statistical and psychometric techniques are not well-suited to aggregating different assessment formats accumulated over time. Some of the strengths of programmatic assessment are also vulnerabilities viewed through this lens. While emphasis is often placed on data richness and considered input of qualified experts, committees reasonably wish for practical, defensible solutions to these challenges. Methods: We draw upon on existing literature regarding Bayesian Networks (BN), noting their utility and application in educational systems. We provide illustrative examples of how they could potentially be used in contexts that embed programmatic principles. We show a simple BN for a knowledge domain before presenting a full-scale 'proof of concept' BN to support committee decisions. We zoom in on one 'node' to demonstrate the capacity of incorporating disparate evidence throughout the network. Conclusions: Bayesian Networks offer an approach that is theoretically well-supported for programmatic assessment. They can aid committees in managing evidence accumulation, help them make inferences under conditions of uncertainty, and buttress decisions by adding a layer of defensibility to the process. They are a pragmatic tool adding value to the programmatic space by applying a complementary statistical framework. We see four major benefits of BNs in programmatic assessment: BNs allow for visual capturing of evidentiary arguments by committees during decision-making; 'recommendations' from probabilistic pathways can be used by committees to confirm their qualitative judgments; BNs can ensure precedents are maintained and consistency occurs over time; and the imperative to capture data richness is maintained without resorting to questionable methodological strategies such as adding qualitatively different things together. Further research into their feasibility and robustness in practice is warranted.

A Case Study in Knowledge Discovery and Elicitation in an Intelligent Tutoring Application

Learning bayesian networks using domain knowledge: An empirical study

Incorporating expert knowledge when learning Bayesian network structure: A medical case study

Dynamic Knowledge Inference Based on Bayesian Network Learning

Extraction of Belief Knowledge from a Relational Database for Quantitative Bayesian Network Inference

Fast & Efficient Learning of Bayesian Networks from Data: Knowledge Discovery and Causality

A survey of Bayesian Network structure learning

A Bayesian Method for Constructing Bayesian Belief Networks from Databases

Bayes Nets in Educational Assessment: Where Do the Numbers Come From?

Causal discovery using dynamically requested knowledge

BARD: A Structured Technique for Group Elicitation of Bayesian Networks to Support Analytic Reasoning

An Introduction to Bayesian Knowledge Tracing with pyBKT

Data-Intensive Learning Of Uncertain Knowledge

Efficient Heuristics for Learning Bayesian Network from Labeled and Unlabeled Data

The potential use of Bayesian Networks to support committee decisions in programmatic assessment

Recognition Networks for Approximate Inference in BN20 Networks

Review of intelligent tutoring systems using bayesian approach

Adaptive user support in educational environments: A Bayesian Network approach

Auto Diagnosing: An Intelligent Assessment System Based on Bayesian Networks

Equity and Fairness of Bayesian Knowledge Tracing

Knowledge tracing: Modeling the acquisition of procedural knowledge