Abstract:Our purpose, is to put forward a change in the paradigm of testing by generalizing a very natural idea exposed by Morris DeGroot (1975) aiming to an approach that is attractive to all schools of statistics, in a procedure better suited for the needs of science. DeGroot's seminal idea is to base testing statistical hypothesis on minimizing the weighted sum of type I plus type II error instead of of the prevailing paradigm which is fixing type I error and minimizing type II error. DeGroot's result is that in simple vs simple hypothesis the optimal criterion is to reject, according to the likelihood ratio as the evidence (ordering) statistics using a fixed threshold value, instead of a fixed tail probability. By defining expected type I and type II errors, we generalize DeGroot's approach and find that the optimal region is defined by the ratio of evidences, that is, averaged likelihoods (with respect to a prior measure) and a threshold fixed. This approach yields an optimal theory in complete generality, which the Classical Theory of Testing does not. This can be seen as a Bayes-Non-Bayes compromise: the criteria (weighted sum of type I and type II errors) is Frequentist, but the test criterion is the ratio of marginalized likelihood, which is Bayesian. We give arguments, to push the theory still further, so that the weighting measures (priors)of the likelihoods does not have to be proper and highly informative, but just predictively matched, that is that predictively matched priors, give rise to the same evidence (marginal likelihoods) using minimal (smallest) training samples. The theory that emerges, similar to the theories based on Objective Bayes approaches, is a powerful response to criticisms of the prevailing approach of hypothesis testing, see for example Ioannidis (2005) and Siegfried (2010) among many others.

On the Equivalence between Bayesian and Classical Hypothesis Testing

Bayesian Hypothesis Tests Using Nonparametric Statistics

On Bayes factors for hypothesis tests

Why bother with Bayesian t-tests?

Error Probabilities in Default Bayesian Hypothesis Testing

Bayesian hypothesis tests with diffuse priors: Can we have our cake and eat it too?

Bayes Factor in One-Sample Tests of Means with a Sensitivity Analysis: A Discussion of Separate Prior Distributions

Confidences in Hypotheses

Empirical Bayes factors for common hypothesis tests

A new approach to Bayesian hypothesis testing

The Bayesian Hypothesis Testing of a Few Non-normal Parameters

A Good check on the Bayes factor

On Some Assumptions of the Null Hypothesis Statistical Testing

An Objective and Robust Bayes Factor for the Hypothesis Test One Sample and Two Population Means

A puzzle of proportions: Two popular Bayesian tests can yield dramatically different conclusions

A class of sequential multi-hypothesis tests

The case for balanced hypothesis tests and equal-tailed confidence intervals

Bayesian Hypothesis Testing in Latent Variable Models

Quantization of Prior Probabilities for Hypothesis Testing

Changing the paradigm of fixed significance levels: Testing Hypothesis by Minimizing Sum of Errors Type I and Type II

The expected demise of the Bayes factor