Bayesian Statistics Then and Now

Andrew Gelman

DOI: https://doi.org/10.1214/10-STS308B

2010-12-07

Abstract:Discussion of "The Future of Indirect Evidence" by Bradley Efron [<a class="link-https" data-arxiv-id="1012.1161" href="https://arxiv.org/abs/1012.1161">arXiv:1012.1161</a>]

Methodology

What problem does this paper attempt to address?

The Future of Indirect Evidence

Bradley Efron

DOI: https://doi.org/10.1214/09-STS308

2010-12-08

Abstract:Familiar statistical tests and estimates are obtained by the direct observation of cases of interest: a clinical trial of a new drug, for instance, will compare the drug's effects on a relevant set of patients and controls. Sometimes, though, indirect evidence may be temptingly available, perhaps the results of previous trials on closely related drugs. Very roughly speaking, the difference between direct and indirect statistical evidence marks the boundary between frequentist and Bayesian thinking. Twentieth-century statistical practice focused heavily on direct evidence, on the grounds of superior objectivity. Now, however, new scientific devices such as microarrays routinely produce enormous data sets involving thousands of related situations, where indirect evidence seems too important to ignore. Empirical Bayes methodology offers an attractive direct/indirect compromise. There is already some evidence of a shift toward a less rigid standard of statistical objectivity that allows better use of indirect evidence. This article is basically the text of a recent talk featuring some examples from current practice, with a little bit of futuristic speculation.

Methodology
Bayesian Models and Methods in Public Policy and Government Settings

S. Fienberg

DOI: https://doi.org/10.1214/10-STS331

2011-05-01

Abstract:Starting with the neo-Bayesian revival of the 1950s, many statisticians argued that it was inappropriate to use Bayesian methods, and in particular subjective Bayesian methods in governmental and public policy settings because of their reliance upon prior distributions. But the Bayesian framework often provides the primary way to respond to questions raised in these settings and the numbers and diversity of Bayesian applications have grown dramatically in recent years. Through a series of examples, both historical and recent, we argue that Bayesian approaches with formal and informal assessments of priors AND likelihood functions are well accepted and should become the norm in public settings. Our examples include census-taking and small area estimation, US election night forecasting, studies reported to the US Food and Drug Administration, assessing global climate change, and measuring potential declines in disability among the elderly.

Political Science,Mathematics,Economics
Bayesian Statistical Pragmatism

Andrew Gelman

DOI: https://doi.org/10.1214/11-STS337C

2011-06-16

Abstract:Discussion of "Statistical Inference: The Big Picture" by R. E. Kass [<a class="link-https" data-arxiv-id="1106.2895" href="https://arxiv.org/abs/1106.2895">arXiv:1106.2895</a>]

Methodology
Empirical Bayes Methods, Evidentialism, and the Inferential Roles They Play

Shetty,Brittan,Bandyopadhyay

DOI: https://doi.org/10.3390/e26100859

IF: 2.738

2024-10-13

Entropy

Abstract:Empirical Bayes-based Methods (EBM) is an increasingly popular form of Objective Bayesianism (OB). It is identified in particular with the statistician Bradley Efron. The main aims of this paper are, first, to describe and illustrate its main features and, second, to locate its role by comparing it with two other statistical paradigms, Subjective Bayesianism (SB) and Evidentialism. EBM's main formal features are illustrated in some detail by schematic examples. The comparison between what Efron calls their underlying "philosophies" is by way of a distinction made between confirmation and evidence. Although this distinction is sometimes made in the statistical literature, it is relatively rare and never to the same point as here. That is, the distinction is invariably spelled out intra- and not inter-paradigmatically solely in terms of one or the other accounts. The distinction made in this paper between confirmation and evidence is illustrated by two well-known statistical paradoxes: the base-rate fallacy and Popper's paradox of ideal evidence. The general conclusion reached is that each of the paradigms has a basic role to play and all are required by an adequate account of statistical inference from a technically informed and fine-grained philosophical perspective.

physics, multidisciplinary
Bayesian astrostatistics: a backward look to the future

Thomas J. Loredo

DOI: https://doi.org/10.1007/978-1-4614-3508-2_2

2012-08-30

Abstract:This perspective chapter briefly surveys: (1) past growth in the use of Bayesian methods in astrophysics; (2) current misconceptions about both frequentist and Bayesian statistical inference that hinder wider adoption of Bayesian methods by astronomers; and (3) multilevel (hierarchical) Bayesian modeling as a major future direction for research in Bayesian astrostatistics, exemplified in part by presentations at the first ISI invited session on astrostatistics, commemorated in this volume. It closes with an intentionally provocative recommendation for astronomical survey data reporting, motivated by the multilevel Bayesian perspective on modeling cosmic populations: that astronomers cease producing catalogs of estimated fluxes and other source properties from surveys. Instead, summaries of likelihood functions (or marginal likelihood functions) for source properties should be reported (not posterior probability density functions), including nontrivial summaries (not simply upper limits) for candidate objects that do not pass traditional detection thresholds.

Instrumentation and Methods for Astrophysics,Data Analysis, Statistics and Probability,Applications
Bayesian statistics in educational research: a look at the current state of affairs

Christoph König,Rens van de Schoot

DOI: https://doi.org/10.1080/00131911.2017.1350636

2017-07-25

Educational Review

Abstract:The ability of a scientific discipline to build cumulative knowledge depends on its predominant method of data analysis. A steady accumulation of knowledge requires approaches which allow researchers to consider results from comparable prior research. Bayesian statistics is especially relevant for establishing a cumulative scientific discipline, because the incorporation of background (or prior) knowledge is fundamentally anchored in its basic principles. The aim of the current systematic review is to provide insights into the current state of methodological affairs in educational research, with a focus on Bayesian statistics and the use of prior information. An analysis of publication histories of the 224 educational journals currently listed in the Thomson Reuters Journal Citation Report 2015 indicates that Bayesian statistics is primarily used to solve methodological problems, rather than used to build cumulative knowledge based on a combination of study results with comparable prior research. The utilisation of Bayesian statistics is motivated by its flexibility: models are estimated which would not be estimable with frequentist approaches, thus expanding the methodological repertoire of educational researchers and producing knowledge which otherwise would not have been available. Lastly, the predominant use of noninformative prior distributions indicates that one of the biggest advantages of Bayesian statistics, namely the combination of study results with comparable prior research, remains underutilised in educational research. Practical implications of these findings for educational research are illustrated and discussed.

education & educational research
Fully Bayesian Forecasts with Evidence Networks

T. Gessey-Jones,W. J. Handley

2024-05-23

Abstract:Sensitivity forecasts inform the design of experiments and the direction of theoretical efforts. To arrive at representative results, Bayesian forecasts should marginalize their conclusions over uncertain parameters and noise realizations rather than picking fiducial values. However, this is typically computationally infeasible with current methods for forecasts of an experiment's ability to distinguish between competing models. We thus propose a novel simulation-based methodology capable of providing expedient and rigorous Bayesian model comparison forecasts without relying on restrictive assumptions.

Instrumentation and Methods for Astrophysics,Cosmology and Nongalactic Astrophysics,General Relativity and Quantum Cosmology
Bayes, Jeffreys, Prior Distributions and the Philosophy of Statistics

Andrew Gelman

DOI: https://doi.org/10.1214/09-STS284D

2010-01-18

Abstract:Discussion of "Harold Jeffreys's Theory of Probability revisited," by Christian Robert, Nicolas Chopin, and Judith Rousseau, for Statistical Science [<a class="link-https" data-arxiv-id="0804.3173" href="https://arxiv.org/abs/0804.3173">arXiv:0804.3173</a>]

Methodology
Bayesian data analysis.

J. Kruschke

DOI: https://doi.org/10.1002/wcs.72

2010-09-01

Abstract:Bayesian methods have garnered huge interest in cognitive science as an approach to models of cognition and perception. On the other hand, Bayesian methods for data analysis have not yet made much headway in cognitive science against the institutionalized inertia of 20th century null hypothesis significance testing (NHST). Ironically, specific Bayesian models of cognition and perception may not long endure the ravages of empirical verification, but generic Bayesian methods for data analysis will eventually dominate. It is time that Bayesian data analysis became the norm for empirical methods in cognitive science. This article reviews a fatal flaw of NHST and introduces the reader to some benefits of Bayesian data analysis. The article presents illustrative examples of multiple comparisons in Bayesian analysis of variance and Bayesian approaches to statistical power. Copyright © 2010 John Wiley & Sons, Ltd. For further resources related to this article, please visit the WIREs website.

Computer Science,Medicine,Psychology
Comment: Microarrays, Empirical Bayes and the Two-Groups Model

Carl N. Morris

DOI: https://doi.org/10.48550/arXiv.0808.0597

2008-08-05

Methodology

Abstract:Brad Efron's paper [arXiv:0808.0572] has inspired a return to the ideas behind Bayes, frequency and empirical Bayes. The latter preferably would not be limited to exchangeable models for the data and hyperparameters. Parallels are revealed between microarray analyses and profiling of hospitals, with advances suggesting more decision modeling for gene identification also. Then good multilevel and empirical Bayes models for random effects should be sought when regression toward the mean is anticipated.
There is Nothing Magical about Bayesian Statistics: An Introduction to Epistemic Probabilities in Data Analysis for Psychology Starters

Wojciech Świątkowski,Antonin Carrier

DOI: https://doi.org/10.1080/01973533.2020.1792297

2020-07-25

Basic and Applied Social Psychology

Abstract:This paper is a reader-friendly introduction to Bayesian inference applied to psychological science. We begin by explaining the difference between frequentist and epistemic interpretations of probability that underpin respectively frequentist and Bayesian statistics. We use a concrete example—a student wondering whether s/he carries the virus statisticus malignum—to explain how both approaches are different one from another. We illustrate Bayesian inference with intuitive examples, before introducing the mathematical framework. Different schools of thoughts and recommendations are discussed to illustrate how to use priors in Bayes Factor testing. We discuss how psychology could benefit from a greater reliance on Bayesian methods. Finally, we illustrate how to compute Bayes Factors analyses with real data and provide the R code.

psychology, social
Bayesian statistics for clinical research

Ewan C Goligher,Anna Heath,Michael O Harhay

DOI: https://doi.org/10.1016/S0140-6736(24)01295-9

2024-09-14

Abstract:Frequentist and Bayesian statistics represent two differing paradigms for the analysis of data. Frequentism became the dominant mode of statistical thinking in medical practice during the 20th century. The advent of modern computing has made Bayesian analysis increasingly accessible, enabling growing use of Bayesian methods in a range of disciplines, including medical research. Rather than conceiving of probability as the expected frequency of an event (purported to be measurable and objective), Bayesian thinking conceives of probability as a measure of strength of belief (an explicitly subjective concept). Bayesian analysis combines previous information (represented by a mathematical probability distribution, the prior) with information from the study (the likelihood function) to generate an updated probability distribution (the posterior) representing the information available for clinical decision making. Owing to its fundamentally different conception of probability, Bayesian statistics offers an intuitive, flexible, and informative approach that facilitates the design, analysis, and interpretation of clinical trials. In this Review, we provide a brief account of the philosophical and methodological differences between Bayesian and frequentist approaches and survey the use of Bayesian methods for the design and analysis of clinical research.
Harold Jeffreys's Theory of Probability Revisited

Christian P. Robert,Nicolas Chopin,Judith Rousseau

DOI: https://doi.org/10.1214/09-STS284

2010-01-18

Abstract:Published exactly seventy years ago, Jeffreys's Theory of Probability (1939) has had a unique impact on the Bayesian community and is now considered to be one of the main classics in Bayesian Statistics as well as the initiator of the objective Bayes school. In particular, its advances on the derivation of noninformative priors as well as on the scaling of Bayes factors have had a lasting impact on the field. However, the book reflects the characteristics of the time, especially in terms of mathematical rigor. In this paper we point out the fundamental aspects of this reference work, especially the thorough coverage of testing problems and the construction of both estimation and testing noninformative priors based on functional divergences. Our major aim here is to help modern readers in navigating in this difficult text and in concentrating on passages that are still relevant today.

Statistics Theory,History and Overview
Philosophy and the practice of Bayesian statistics

Andrew Gelman,Cosma Rohilla Shalizi

DOI: https://doi.org/10.1111/j.2044-8317.2011.02037.x

2011-06-29

Abstract:A substantial school in the philosophy of science identifies Bayesian inference with inductive inference and even rationality as such, and seems to be strengthened by the rise and practical success of Bayesian statistics. We argue that the most successful forms of Bayesian statistics do not actually support that particular philosophy but rather accord much better with sophisticated forms of hypothetico-deductivism. We examine the actual role played by prior distributions in Bayesian models, and the crucial aspects of model checking and model revision, which fall outside the scope of Bayesian confirmation theory. We draw on the literature on the consistency of Bayesian updating and also on our experience of applied work in social science. Clarity about these matters should benefit not just philosophy of science, but also statistical practice. At best, the inductivist view has encouraged researchers to fit and compare models without checking them; at worst, theorists have actively discouraged practitioners from performing model checking because it does not fit into their framework.

Statistics Theory,Data Analysis, Statistics and Probability
Bayesian inference: More than Bayes's theorem

Thomas J. Loredo,Robert L. Wolpert

2024-06-29

Abstract:Bayesian inference gets its name from *Bayes's theorem*, expressing posterior probabilities for hypotheses about a data generating process as the (normalized) product of prior probabilities and a likelihood function. But Bayesian inference uses all of probability theory, not just Bayes's theorem. Many hypotheses of scientific interest are *composite hypotheses*, with the strength of evidence for the hypothesis dependent on knowledge about auxiliary factors, such as the values of nuisance parameters (e.g., uncertain background rates or calibration factors). Many important capabilities of Bayesian methods arise from use of the law of total probability, which instructs analysts to compute probabilities for composite hypotheses by *marginalization* over auxiliary factors. This tutorial targets relative newcomers to Bayesian inference, aiming to complement tutorials that focus on Bayes's theorem and how priors modulate likelihoods. The emphasis here is on marginalization over parameter spaces -- both how it is the foundation for important capabilities, and how it may motivate caution when parameter spaces are large. Topics covered include the difference between likelihood and probability, understanding the impact of priors beyond merely shifting the maximum likelihood estimate, and the role of marginalization in accounting for uncertainty in nuisance parameters, systematic error, and model misspecification.

Methodology,Instrumentation and Methods for Astrophysics
Bayesian questions with frequentist answers

Alan H. Guth,Mohammad Hossein Namjoo

2023-08-31

Abstract:The two statistical methods, namely the frequentist and the Bayesian methods, are both commonly used for probabilistic inference in many scientific situations. However, it is not straightforward to interpret the result of one approach in terms of the concepts of the other. In this paper we explore the possibility of finding a Bayesian significance for the frequentist's main object of interest, the $p$-value, which is the probability assigned to the proposition -- which we call the {\it extremity proposition} -- that a measurement will result in a value that is at least as extreme as the value that was actually obtained. To make contact with the frequentist language, the Bayesian can choose to update probabilities based on the {\it extremity proposition}, which is weaker than the standard Bayesian update proposition, which uses the actual observed value. We then show that the posterior probability (or probability density) of a theory is equal to the prior probability (or probability density) multiplied by the ratio of the $p$-value for the data obtained, given that theory, to the mean $p$-value -- averaged over all theories weighted by their prior probabilities. Thus, we provide frequentist answers to Bayesian questions. Our result is generic -- it does not rely on restrictive assumptions about the situation under consideration or specific properties of the likelihoods or the priors.

Data Analysis, Statistics and Probability,Cosmology and Nongalactic Astrophysics,General Relativity and Quantum Cosmology,High Energy Physics - Phenomenology,High Energy Physics - Theory,Other Statistics
Discussion of: Bayesian views of an archaeological find

Joseph B. Kadane

DOI: https://doi.org/10.48550/arXiv.0804.0102

2008-04-01

Applications

Abstract:Discussion of ``Statistical analysis of an archeological find'' by Andrey Feuerverger [arXiv:0804.0079]
Bayesian data analysis

Andrew Gelman, John Carlin, Hal Stern, Donald B Rubin

2004-01-01

Abstract:
Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications

Eric-Jan Wagenmakers,Maarten Marsman,Tahira Jamil,Alexander Ly,Josine Verhagen,Jonathon Love,Ravi Selker,Quentin F. Gronau,Martin Šmíra,Sacha Epskamp,Dora Matzke,Jeffrey N. Rouder,Richard D. Morey

DOI: https://doi.org/10.3758/s13423-017-1343-3

2017-08-04

Abstract:Bayesian parameter estimation and Bayesian hypothesis testing present attractive alternatives to classical inference using confidence intervals and p values. In part I of this series we outline ten prominent advantages of the Bayesian approach. Many of these advantages translate to concrete opportunities for pragmatic researchers. For instance, Bayesian hypothesis testing allows researchers to quantify evidence and monitor its progression as data come in, without needing to know the intention with which the data were collected. We end by countering several objections to Bayesian hypothesis testing. Part II of this series discusses JASP, a free and open source software program that makes it easy to conduct Bayesian estimation and testing for a range of popular statistical scenarios (Wagenmakers et al. this issue).

psychology, experimental, mathematical

Bayesian Statistics Then and Now

The Future of Indirect Evidence

Bayesian Models and Methods in Public Policy and Government Settings

Bayesian Statistical Pragmatism

Empirical Bayes Methods, Evidentialism, and the Inferential Roles They Play

Bayesian astrostatistics: a backward look to the future

Bayesian statistics in educational research: a look at the current state of affairs

Fully Bayesian Forecasts with Evidence Networks

Bayes, Jeffreys, Prior Distributions and the Philosophy of Statistics

Bayesian data analysis.

Comment: Microarrays, Empirical Bayes and the Two-Groups Model

There is Nothing Magical about Bayesian Statistics: An Introduction to Epistemic Probabilities in Data Analysis for Psychology Starters

Bayesian statistics for clinical research

Harold Jeffreys's Theory of Probability Revisited

Philosophy and the practice of Bayesian statistics

Bayesian inference: More than Bayes's theorem

Bayesian questions with frequentist answers

Discussion of: Bayesian views of an archaeological find

Bayesian data analysis

Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications