Abstract:Over the past decade, the field of forensic science has received recommendations from the National Research Council of the U.S. National Academy of Sciences, the U.S. National Institute of Standards and Technology, and the U.S. President's Council of Advisors on Science and Technology to study the validity and reliability of forensic analyses. More specifically, these committees recommend estimation of the rates of occurrence of erroneous conclusions drawn from forensic analyses. "Black box" studies for the various subjective feature-based comparison methods are intended for this purpose. In general, "black box" studies often have unbalanced designs, comparisons that are not independent, and missing data. These aspects pose difficulty in the analysis of the results and are often ignored. Instead, interpretation of the data relies on methods that assume independence between observations and a balanced experiment. Furthermore, all of these projects are interpreted within the frequentist framework and result in point estimates associated with confidence intervals that are confusing to communicate and understand. We propose to use an existing likelihood-free Bayesian inference method, called Approximate Bayesian Computation (ABC), that is capable of handling unbalanced designs, dependencies among the observations, and missing data. ABC allows for studying the parameters of interest without recourse to incoherent and misleading measures of uncertainty such as confidence intervals. By taking into account information from all decision categories for a given examiner and information from the population of examiners, our method also allows for quantifying the risk of error for the given examiner, even when no error has been recorded for that examiner. We illustrate our proposed method by reanalysing the results of the "Noblis Black Box" study by Ulery et al. in 2011.

A Unified Bayesian Framework for Modeling Measurement Error in Multinomial Data

Robust Bayesian Inference for Berkson and Classical Measurement Error Models

Seemingly Unrelated Regression with Measurement Error: Estimation via Markov chain Monte Carlo and Mean Field Variational Bayes Approximation

Implementing measurement error models with mechanistic mathematical models in a likelihood-based framework for estimation, identifiability analysis, and prediction in the life sciences

Bayesian Multiple Imputation for Assay Data Subject to Measurement Error

A general algorithm for error-in-variables regression modelling using Monte Carlo expectation maximization

Analysis of Structural Health Monitoring Data with Correlated Measurement Error by Bayesian System Identification: Theory and Application.

Mesoscopic Bayesian Inference by Solvable Models

Bayesian estimation for longitudinal data in a joint model with HPCs

Bayesian Latent-Class Mixed-Effect Hybrid Models for Dyadic Longitudinal Data with Non-Ignorable Dropouts.

A Bayesian approach for the analysis of error rate studies in forensic science

Measurement error models with zero inflation and multiple sources of zeros, with applications to hard zeros

Modeling complex measurement error in microbiome experiments

Bayesian Measurement Error Correction in Structured Additive Distributional Regression with an Application to the Analysis of Sensor Data on Soil-Plant Variability

Learning Models from Data with Measurement Error: Tackling Underreporting

Bayesian Sample Size Computations inComplex Models with Application to RepeatedMeasures Random E ects Model Design

A Fully Nonparametric Modelling Approach to Binary Regression

A Bayesian Zero-Inflated Dirichlet-Multinomial Regression Model for Multivariate Compositional Count Data

Interrogating Random and Systematic Measurement Error in Morphometric Data

Semiparametric Bayesian Inference on Generalized Linear Measurement Error Models