Abstract:The likelihood ratio paradigm for quantifying the strength of evidence has been researched in many fields of forensic science. Within this paradigm, score-based approaches for estimating likelihood ratios are becoming more prevalent in the forensic science literature. In this study, a score-based approach for estimating likelihood ratios is implemented for linguistic text evidence. Text data are represented via a bag-of-words model with the Z-score normalised relative frequencies of selected most-frequent words (the number of the most-frequent words = N), and the Euclidean, Manhattan and Cosine distance measures are trialled as the score-generating functions for comparing paired text samples. The score-to-likelihood-ratio conversion model was built using a common source method, and the best fitting model was selected from the parametric models of the Normal, Log-normal, Gamma and Weibull distributions. With the Amazon Product Data Authorship Verification Corpus, two groups of documents (each group including documents of approximately 700, 1400 and 2100 words) were synthesised for each author, allowing 720 same-author comparisons and 517,680 different-author comparisons to test the validity of the system. A series of experiments was conducted using combinations of the following conditions: the three score functions, the different values of N for the feature vector and the different document lengths. The validity of the system was assessed using the log-likelihood-ratio cost (Cllr), and the strength of the derived likelihood ratios was charted in the form of Tippett plots. It was demonstrated that 1) the Cosine measure consistently outperforms the other measures-the best performance is achieved with N = 260, regardless of the document length (e.g., Cllr values of 0.70640, 0.45314 and 0.30692, respectively, for 700, 1400 and 2100 words)-and 2) the derived likelihood ratios are very well calibrated irrespective of the distance measures and document lengths. A follow-up experiment showed that the described score-based approach is relatively robust and stable for a limited quantity of background data. The derived likelihood ratios that were estimated separately to the three distance measures were logistic regression fused; and the fusion achieved a further improvement in performance-for example, a Cllr of 0.23494 for 2100 words. This study demonstrates the possibility of designing likelihood ratio-based systems that discriminate between same-author and different-author documents.

On the Equivalence of a Likelihood Ratio of Drasgow, Levine, and Zickar (1996) and the Statistic Based on the Neyman-Pearson Lemma of Belov (2016)

The Lack of Robustness of a Statistic Based on the Neyman-Pearson Lemma to Violations of Its Underlying Assumptions

Detection of Item Preknowledge Using Likelihood Ratio Test and Score Test

Non-parametric Tests for the Tail Equivalence Via Empirical Likelihood

Higher-Order Asymptotics and Its Application to Testing the Equality of the Examinee Ability Over Two Sets of Items

On the E ffi ciency of Generalized Likelihood Ratio Tests

Calculation of forensic likelihood ratios: Use of Monte Carlo simulations to compare the output of score-based approaches with true likelihood-ratio values

The Statistical Accuracy of Neural Posterior and Likelihood Estimation

Which Statistic Should Be Used to Detect Item Preknowledge When the Set of Compromised Items Is Known?

Saddlepoint approximations for likelihood ratio like statistics with applications to permutation tests

Sieve empirical likelihood ratio tests for nonparametric functions

A comparison of Bayesian and score methods for interval estimates of positive/negative likelihood ratios in support of diagnostic device performance evaluation

Assumption-lean falsification tests of rate double-robustness of double-machine-learning estimators

Asymptotic bayes optimality under sparsity for equicorrelated multivariate normal test statistics

Analysis of two Binomial Proportions in Non-inferiority Confirmatory Trials

Inference on order restricted means of inverse Gaussian populations under heteroscedasticity

A Likelihood-based Alternative to Null Hypothesis Significance Testing

Likelihood-ratio test statistic for the finite-sample case in nonlinear ordinary differential equation models

Robust and efficient Breusch-Pagan test-statistic: an application of the beta-score Lagrange multipliers test for non-identically distributed individuals

Performing Likelihood Ratio Tests with Multiply-Imputed Data Sets

Score-based likelihood ratios for linguistic text evidence with a bag-of-words model