Abstract:Over the past decade, the field of forensic science has received recommendations from the National Research Council of the U.S. National Academy of Sciences, the U.S. National Institute of Standards and Technology, and the U.S. President's Council of Advisors on Science and Technology to study the validity and reliability of forensic analyses. More specifically, these committees recommend estimation of the rates of occurrence of erroneous conclusions drawn from forensic analyses. "Black box" studies for the various subjective feature-based comparison methods are intended for this purpose. In general, "black box" studies often have unbalanced designs, comparisons that are not independent, and missing data. These aspects pose difficulty in the analysis of the results and are often ignored. Instead, interpretation of the data relies on methods that assume independence between observations and a balanced experiment. Furthermore, all of these projects are interpreted within the frequentist framework and result in point estimates associated with confidence intervals that are confusing to communicate and understand. We propose to use an existing likelihood-free Bayesian inference method, called Approximate Bayesian Computation (ABC), that is capable of handling unbalanced designs, dependencies among the observations, and missing data. ABC allows for studying the parameters of interest without recourse to incoherent and misleading measures of uncertainty such as confidence intervals. By taking into account information from all decision categories for a given examiner and information from the population of examiners, our method also allows for quantifying the risk of error for the given examiner, even when no error has been recorded for that examiner. We illustrate our proposed method by reanalysing the results of the "Noblis Black Box" study by Ulery et al. in 2011.

A Bayesian framework to evaluate evidence in cases of alleged cheating with secret codes in sports

A Bayesian Approach to In-Game Win Probability in Soccer

A compound framework for sports results prediction: A football case study

A Bayesian approach for the analysis of error rate studies in forensic science

Information-Theoretic Scoring Rules to Learn Additive Bayesian Network Applied to Epidemiology

Bayesian reasoning and the prior in court: not legally normative but unavoidable

Evaluating evidence in linked crimes with multiple offenders

The Role of the Bayes Factor in the Evaluation of Evidence

State Information in Bayesian Games

A generalization of the Bradley–Terry model for draws in chess with an application to collusion

Algorithmic Persuasion with Evidence

Using evidence to make decisions

A Good check on the Bayes factor

Bayesian Persuasion under Ex Ante and Ex Post Constraints

Bayesian Reasoning and Evidence Communication

Modeling Other Players with Bayesian Beliefs for Games with Incomplete Information

Bayesian evidence synthesis as a flexible alternative to meta-analysis: A simulation study and empirical demonstration

A hybrid formal theory of arguments, stories and criminal evidence

A State-Space Perspective on Modelling and Inference for Online Skill Rating

A Bayesian inference approach for determining player abilities in football

Discounting testimony with the argument ad hominem and a Bayesian congruent prior model.