Abstract:The validity of a probabilistic genotyping (PG) system is typically demonstrated by following international guidelines for the developmental and internal validation of PG software. These guidelines mainly focus on discriminatory power. Very few studies have reported with metrics that depend on calibration of likelihood ratio (LR) systems. In this study, discriminatory power as well as various calibration metrics, such as Empirical Cross-Entropy (ECE) plots, pool adjacent violator (PAV) plots, log likelihood ratio cost (Cllr and Cllrcal), fiducial calibration discrepancy plots, and Turing expectation were examined using the publicly-available PROVEDIt dataset. The aim was to gain deeper insight into the performance of a variety of PG software in the lower LR ranges (~LR 1-10,000), with focus on DNAStatistX and EuroForMix which use maximum likelihood estimation (MLE). This may be a driving force for the end users to reconsider current LR thresholds for reporting. In previous studies, overstated low LRs were observed for these PG software. However, applying (arbitrarily) high LR thresholds for reporting wastes relevant evidential value. This study demonstrates, based on calibration performance, that previously reported LR thresholds can be lowered or even discarded. Considering LRs >1, there was no evidence for miscalibration performance above LR ~1,000 when using Fst 0.01. Below this LR value, miscalibration was observed. Calibration performance generally improved with the use of Fst 0.03, but the extent of this was dependent on the dataset: results ranged from miscalibration up to LR ~100 to no evidence of miscalibration alike PG software using different methods to model peak height, HMC and STRmix. This study demonstrates that practitioners using MLE-based models should be careful when low LR ranges are reported, though applying arbitrarily high LR thresholds is discouraged. This study also highlights various calibration metrics that are useful in understanding the performance of a PG system.

Comments arising from WJ Thompson "Uncertainty in probabilistic genotyping of low template DNA A case study comparing STRmix and TrueAllele"

A tale of two PG systems: A comparison of the two most widely used continuous probabilistic genotyping systems in the United States

Complex DNA mixture analysis in a forensic context: evaluating the probative value using a likelihood ratio model

Extending the discussion on inconsistency in forensic decisions and results

A unifying framework for the modelling and analysis of STR DNA samples arising in forensic casework

A collaborative study on the precision of the Markov chain Monte Carlo algorithms used for DNA profile interpretation

On the limitations of probabilistic claims about the probative value of mixed DNA profile evidence

Low LRs obtained from DNA mixtures: On calibration and discrimination performance of probabilistic genotyping software

Variability and additivity of read counts for aSTRs in NGS DNA profiles

Interpreting forensic DNA mixtures: allowing for uncertainty in population substructure and dependence

An exploratory view into allelic drop‐out of sequenced autosomal STRs

A comparison of software for analysis of rare and common short tandem repeat (STR) variation using human genome sequences from clinical and population-based samples

Encoding of low-quality DNA profiles as genotype probability matrices for improved profile comparisons, relatedness evaluation and database searches

Optimizing Analytical Thresholds for Low-Template DNA Analysis: Insights from Multi-Laboratory Negative Controls

[Genotype Reliability of Short Tandem Repeats Typing from Minute DNA].

A straightforward multiallelic significance test for the Hardy-Weinberg equilibrium law

The Usefulness of qPCR Data for Sample Pre-Assessment and Interpretation of Genetic Typing Results

Incorporating individual error rate into association test of unmatched case-control design.

Statistical methods for discrimination of STR genotypes using high resolution melt curve data

A Comparison of Approaches to Account for Uncertainty in Analysis of Imputed Genotypes

Evaluation of the Microhaplotypes Panel for DNA Mixture Analyses.