Abstract:Our purpose, is to put forward a change in the paradigm of testing by generalizing a very natural idea exposed by Morris DeGroot (1975) aiming to an approach that is attractive to all schools of statistics, in a procedure better suited for the needs of science. DeGroot's seminal idea is to base testing statistical hypothesis on minimizing the weighted sum of type I plus type II error instead of of the prevailing paradigm which is fixing type I error and minimizing type II error. DeGroot's result is that in simple vs simple hypothesis the optimal criterion is to reject, according to the likelihood ratio as the evidence (ordering) statistics using a fixed threshold value, instead of a fixed tail probability. By defining expected type I and type II errors, we generalize DeGroot's approach and find that the optimal region is defined by the ratio of evidences, that is, averaged likelihoods (with respect to a prior measure) and a threshold fixed. This approach yields an optimal theory in complete generality, which the Classical Theory of Testing does not. This can be seen as a Bayes-Non-Bayes compromise: the criteria (weighted sum of type I and type II errors) is Frequentist, but the test criterion is the ratio of marginalized likelihood, which is Bayesian. We give arguments, to push the theory still further, so that the weighting measures (priors)of the likelihoods does not have to be proper and highly informative, but just predictively matched, that is that predictively matched priors, give rise to the same evidence (marginal likelihoods) using minimal (smallest) training samples. The theory that emerges, similar to the theories based on Objective Bayes approaches, is a powerful response to criticisms of the prevailing approach of hypothesis testing, see for example Ioannidis (2005) and Siegfried (2010) among many others.

The Bayesian Hypothesis Testing of a Few Non-normal Parameters

Bayesian Hypothesis Tests Using Nonparametric Statistics

A new approach to Bayesian hypothesis testing

A Bayesian Chi-Squared Test for Hypothesis Testing

On the Equivalence between Bayesian and Classical Hypothesis Testing

Frequentist properties of Bayesian inequality tests

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

Bayesian Hypothesis Testing in Latent Variable Models

Posterior-based Wald-type Statistics for Hypothesis Testing

Sub-Gaussian Error Bounds for Hypothesis Testing

Bayesian hypothesis tests with diffuse priors: Can we have our cake and eat it too?

An Entropy-Based Approach for Nonparametrically Testing Simple Probability Distribution Hypotheses

Why bother with Bayesian t-tests?

Towards a theory for testing statistical hypothesis: Multivariate mean with nuisance covariance matrix

Quantization of Prior Probabilities for Hypothesis Testing

A frequentist two-sample test based on Bayesian model selection

Testing the Skewness of Skew-Normal Distribution by Bayes Factors

Bayesian Test of Normality Versus a Dirichlet Process Mixture Alternative

Hypothesis Testing in Gaussian Graphical Models: Novel Goodness-of-Fit Tests and Conditional Randomization Tests

Doublethink: simultaneous Bayesian-frequentist model-averaged hypothesis testing

Changing the paradigm of fixed significance levels: Testing Hypothesis by Minimizing Sum of Errors Type I and Type II