Abstract:In many hypothesis testing applications, we have mixed priors, with well-motivated informative priors for some parameters but not for others. The Bayesian methodology uses the Bayes factor and is helpful for the informative priors, as it incorporates Occam's razor via multiplicity or trials factor in the Look Elsewhere Effect. However, if the prior is not known completely, the frequentist hypothesis test via the false positive rate is a better approach, as it is less sensitive to the prior choice. We argue that when only partial prior information is available, it is best to combine the two methodologies by using the Bayes factor as a test statistic in the frequentist analysis. We show that the standard frequentist likelihood-ratio test statistic corresponds to the Bayes factor with a non-informative Jeffrey's prior. We also show that mixed priors increase the statistical power in frequentist analyses over the likelihood ratio test statistic. We develop an analytic formalism that does not require expensive simulations using a statistical mechanics approach to hypothesis testing in Bayesian and frequentist statistics. We introduce the counting of states in a continuous parameter space using the uncertainty volume as the quantum of the state. We show that both the p-value and Bayes factor can be expressed as energy versus entropy competition. We present analytic expressions that generalize Wilks' theorem beyond its usual regime of validity and work in a non-asymptotic regime. In specific limits, the formalism reproduces existing expressions, such as the p-value of linear models and periodograms. We apply the formalism to an example of exoplanet transits, where multiplicity can be more than $10^7$. We show that our analytic expressions reproduce the p-values derived from numerical simulations.

Bayesian estimation of the number of species from Poisson-Lindley stochastic abundance model using non-informative priors

Correction to: Bayesian estimation of the number of species from Poisson–Lindley stochastic abundance model using non-informative priors

Bayesian Nonparametric Inference for "Species-sampling" Problems

Generalized Species Sampling Priors with Latent Beta reinforcements

Note of Correction: “conjugate Analysis of the Conway-Maxwell-Poisson Distribution”

Marginally Specified Priors for Nonparametric Bayesian Estimation

Empirical Bayes Methods for Prior Estimation in Systems Medicine

Bayesian Quantile Regression Based on the Empirical Likelihood with Spike and Slab Priors

Bayesian analysis of product feature allocation models

Empirical Bayes in Bayesian learning: understanding a common practice

Is non-informative Bayesian analysis appropriate for wildlife management: survival of San Joaquin Kit Fox and declines in amphibian populations

Empirical Bayes estimates of steady-state availability in exponential distribution model

Statistical significance testing for mixed priors: a combined Bayesian and frequentist analysis

Translating predictive distributions into informative priors

Bayes Factor of Zero Inflated Models under Jeffereys Prior

On Non- and Weakly-Informative Priors for the Conway-Maxwell-Poisson (COM-Poisson) Distribution

Nearly minimax empirical Bayesian prediction of independent Poisson observables

Flexible Bayesian Modeling of Counts: Constructing Penalized Complexity Priors

Learning Approximately Objective Priors

Posterior propriety in Bayesian extreme value analyses using reference priors