Bayes = Blackwell, Almost

Mark Whitmeyer

DOI: https://doi.org/10.48550/arXiv.2302.13956

2023-10-20

Abstract:There are updating rules other than Bayes' law that render the value of information positive. I find all of them.

Theoretical Economics

What problem does this paper attempt to address?

Information Validates the Prior: A Theorem on Bayesian Updating and Applications

Navin Kartik,Frances Lee,Wing Suen

DOI: https://doi.org/10.1257/aeri.20200284

2020-07-22

Abstract:We develop a result on expected posteriors for Bayesians with heterogenous priors, dubbed information validates the prior (IVP). Under familiar ordering requirements, Anne expects a (Blackwell) more informative experiment to bring Bob's posterior mean closer to Anne's prior mean. We apply the result in two contexts of games of asymmetric information: voluntary testing or certification, and costly signaling or falsification. IVP can be used to determine how an agent's behavior responds to additional exogenous or endogenous information. We discuss economic implications.

Theoretical Economics
Judicious Judgment Meets Unsettling Updating: Dilation, Sure Loss, and Simpson's Paradox

Ruobin Gong,Xiao-Li Meng

DOI: https://doi.org/10.48550/arXiv.1712.08946

2017-12-24

Statistics Theory

Abstract:Statistical learning using imprecise probabilities is gaining more attention because it presents an alternative strategy for reducing irreplicable findings by freeing the user from the task of making up unwarranted high-resolution assumptions. However, model updating as a mathematical operation is inherently exact, hence updating imprecise models requires the user's judgment in choosing among competing updating rules. These rules often lead to incompatible inferences, and can exhibit unsettling phenomena like dilation, contraction and sure loss, which cannot occur with the Bayes rule and precise probabilities. We revisit a number of famous "paradoxes", including the three prisoners/Monty Hall problem, revealing that a logical fallacy arises from a set of marginally plausible yet jointly incommensurable assumptions when updating the underlying imprecise model. We establish an equivalence between Simpson's paradox and an implicit adoption of a pair of aggregation rules that induce sure loss. We also explore behavioral discrepancies between the generalized Bayes rule, Dempster's rule and the Geometric rule as alternative posterior updating rules for Choquet capacities of order 2. We show that both the generalized Bayes rule and Geometric rule are incapable of updating without prior information regardless of how strong the information in our data is, and that Dempster's rule and the Geometric rule can mathematically contradict each other with respect to dilation and contraction. Our findings show that unsettling updates reflect a collision between the rules' assumptions and the inexactness allowed by the model itself, highlighting the invaluable role of judicious judgment in handling low-resolution information, and the care we must take when applying learning rules to update imprecise probabilities.
Conditional Probability Meets Update Logic.

Johan van Benthem

DOI: https://doi.org/10.1023/a:1025002917675

2003-01-01

Abstract:Dynamic update of information states is a new paradigm in logicalsemantics. But such updates are also a traditional hallmark ofprobabilistic reasoning. This note brings the two perspectives togetherin an update mechanism for probabilities which modifies state spaces.
Justifying the Norms of Inductive Inference

Olav Benjamin Vassend

DOI: https://doi.org/10.1093/bjps/axz041

2022-03-01

The British Journal for the Philosophy of Science

Abstract:Bayesian inference is limited in scope because it cannot be applied in idealized contexts where none of the hypotheses under consideration is true and because it is committed to always using the likelihood as a measure of evidential favouring, even when that is inappropriate. The purpose of this article is to study inductive inference in a very general setting where finding the truth is not necessarily the goal and where the measure of evidential favouring is not necessarily the likelihood. I use an accuracy argument to argue for probabilism and I develop a new kind of argument to argue for two general updating rules, both of which are reasonable in different contexts. One of the updating rules has standard Bayesian updating, Bissiri et al.'s ([<a class="tab-link" href="#axz041-B4">2016</a>]) general Bayesian updating, Douven's ([<a class="tab-link" href="#axz041-B4">2016</a>]) IBE-based updating, and my (<a class="tab-link" href="#axz041-B31">Vassend ([forthcoming]</a>) quasi-Bayesian updating as special cases. The other updating rule is novel. 1. Introduction2. Why Crediblity Functions Should Be Probabilistic3. Deriving the Updating Rules3.1. The combination step3.2. The normalization step3.3. Characterizations of inferential and predictive updating4. Discussion of Inferential and Predictive Updating4.1. The difference between inferential updating and predictive updating4.2. The relationship between inferential updating and other updating procedures5. ConclusionAppendix
Updating information based on generalized credal sets. Part 1: Finite case

Andrey G. Bronevich,Igor N. Rozenberg

DOI: https://doi.org/10.1016/j.fss.2024.108891

IF: 4.462

2024-01-31

Fuzzy Sets and Systems

Abstract:This paper considers updating information and statistical inference based on generalized credal sets. In the first part, we define possible ways of conditioning for generalized credal sets in a finite setting, and its realization based on generalized upper previsions with applications to decision making. We compare the obtained results with the generalized Bayes rule from the theory of imprecise probabilities, with Dempster's rule from the theory of belief functions, and with the models of imprecise probabilities based on the likelihood function proposed by M. Cattaneo.

mathematics, applied,statistics & probability,computer science, theory & methods
State Information in Bayesian Games

Paul Cuff

DOI: https://doi.org/10.48550/arXiv.0911.0874

2009-11-05

Abstract:Two-player zero-sum repeated games are well understood. Computing the value of such a game is straightforward. Additionally, if the payoffs are dependent on a random state of the game known to one, both, or neither of the players, the resulting value of the game has been analyzed under the framework of Bayesian games. This investigation considers the optimal performance in a game when a helper is transmitting state information to one of the players. Encoding information for an adversarial setting (game) requires a different result than rate-distortion theory provides. Game theory has accentuated the importance of randomization (mixed strategy), which does not find a significant role in most communication modems and source coding codecs. Higher rates of communication, used in the right way, allow the message to include the necessary random component useful in games.

Information Theory,Cryptography and Security,Computer Science and Game Theory
Dynamic Update with Probabilities

Johan van Benthem,Jelle Gerbrandy,Barteld Kooi

DOI: https://doi.org/10.1007/s11225-009-9209-y

2009-01-01

Studia Logica

Abstract:Current dynamic-epistemic logics model different types of information change in multi-agent scenarios. We generalize these logics to a probabilistic setting, obtaining a calculus for multi-agent update with three natural slots: prior probability on states, occurrence probabilities in the relevant process taking place, and observation probabilities of events. To match this update mechanism, we present a complete dynamic logic of information change with a probabilistic character. The completeness proof follows a compositional methodology that applies to a much larger class of dynamic-probabilistic logics as well. Finally, we discuss how our basic update rule can be parameterized for different update policies, or learning methods.
Updating information based on generalized credal sets. Part 2: General case

Andrey G. Bronevich,Igor N. Rozenberg

DOI: https://doi.org/10.1016/j.fss.2024.108967

IF: 4.462

2024-04-11

Fuzzy Sets and Systems

Abstract:In the first part of this paper, we have considered: a) generalized credal sets on the powerset of a finite set X ; b) their general definition allowing a generalized credal set to be not convex; c) the justification of basic aggregation rules on generalized credal sets; d) ways of updating information. The second part is devoted to the solutions of the above problems for the general case, when generalized credal sets are defined on arbitrary measurable spaces. In addition, we prove that the conditionals after updating of a generalized credal set could be chosen ratio-equivalent. This result confirms the thesis by R.A. Fisher, which says that the likelihood function is defined uniquely up to a positive coefficient.

mathematics, applied,statistics & probability,computer science, theory & methods
Almost-Bayesian Quadratic Persuasion (Extended Version)

Olivier Massicot,Cédric Langbort

2024-09-24

Abstract:In this article, we relax the Bayesianity assumption in the now-traditional model of Bayesian Persuasion introduced by Kamenica & Gentzkow. Unlike preexisting approaches -- which have tackled the possibility of the receiver (Bob) being non-Bayesian by considering that his thought process is not Bayesian yet known to the sender (Alice), possibly up to a parameter -- we let Alice merely assume that Bob behaves 'almost like' a Bayesian agent, in some sense, without resorting to any specific model. Under this assumption, we study Alice's strategy when both utilities are quadratic and the prior is isotropic. We show that, contrary to the Bayesian case, Alice's optimal response may not be linear anymore. This fact is unfortunate as linear policies remain the only ones for which the induced belief distribution is known. What is more, evaluating linear policies proves difficult except in particular cases, let alone finding an optimal one. Nonetheless, we derive bounds that prove linear policies are near-optimal and allow Alice to compute a near-optimal linear policy numerically. With this solution in hand, we show that Alice shares less information with Bob as he departs more from Bayesianity, much to his detriment.

Computer Science and Game Theory
"Not only defended but also applied": The perceived absurdity of Bayesian inference

Andrew Gelman,Christian P. Robert

DOI: https://doi.org/10.48550/arXiv.1006.5366

2012-06-09

Abstract:The missionary zeal of many Bayesians of old has been matched, in the other direction, by a view among some theoreticians that Bayesian methods are absurd-not merely misguided but obviously wrong in principle. We consider several examples, beginning with Feller's classic text on probability theory and continuing with more recent cases such as the perceived Bayesian nature of the so-called doomsday argument. We analyze in this note the intellectual background behind various misconceptions about Bayesian statistics, without aiming at a complete historical coverage of the reasons for this dismissal.

Statistics Theory,Other Statistics
Non-Bayesian Persuasion

Geoffroy de Clippel,Xu Zhang

DOI: https://doi.org/10.1086/720464

IF: 9.637

2022-04-19

Journal of Political Economy

Abstract:Following Kamenica and Gentzkow, this paper studies persuasion as an information design problem. We investigate how mistakes in probabilistic inference impact optimal persuasion. The concavification method is shown to extend naturally to a large class of belief updating rules, which we identify and characterize. This class comprises many non-Bayesian models discussed in the literature. We apply this new technique to gain insight into the revelation principle, the ranking of updating rules, when persuasion is beneficial to the sender, and when it is detrimental to the receiver. Our key result also extends to shed light on the question of robust persuasion.

economics
Blackwell's Informativeness Ranking with Uncertainty-Averse Preferences

Jian Li,Junjie Zhou

DOI: https://doi.org/10.1016/j.geb.2016.01.009

IF: 1.265

2016-01-01

Games and Economic Behavior

Abstract:Blackwell, 1951, Blackwell, 1953 proposes an informativeness ranking of experiments: experiment I is more Blackwell-informative than experiment II if and only if the value of experiment I is higher than that of experiment II for all expected-utility maximizers. Under commitment and reduction, our main theorem shows that Blackwell equivalence holds for all convex and strongly monotone preferences—i.e., uncertainty-averse preferences (Cerreia-Vioglio et al., 2011b), which nest most ambiguity-averse preferences commonly used in applications as special cases.
Conditional Belief, Knowledge and Probability

Jan van Eijck,Kai Li

DOI: https://doi.org/10.48550/arXiv.1707.08744

2017-07-27

Logic in Computer Science

Abstract:A natural way to represent beliefs and the process of updating beliefs is presented by Bayesian probability theory, where belief of an agent a in P can be interpreted as a considering that P is more probable than not P. This paper attempts to get at the core logical notion underlying this. The paper presents a sound and complete neighbourhood logic for conditional belief and knowledge, and traces the connections with probabilistic logics of belief and knowledge. The key notion in this paper is that of an agent a believing P conditionally on having information Q, where it is assumed that Q is compatible with what a knows. Conditional neighbourhood logic can be viewed as a core system for reasoning about subjective plausibility that is not yet committed to an interpretation in terms of numerical probability. Indeed, every weighted Kripke model gives rise to a conditional neighbourhood model, but not vice versa. We show that our calculus for conditional neighbourhood logic is sound but not complete for weighted Kripke models. Next, we show how to extend the calculus to get completeness for the class of weighted Kripke models. Neighbourhood models for conditional belief are closed under model restriction (public announcement update), while earlier neighbourhood models for belief as `willingness to bet' were not. Therefore the logic we present improves on earlier neighbourhood logics for belief and knowledge. We present complete calculi for public announcement and for publicly revealing the truth value of propositions using reduction axioms. The reductions show that adding these announcement operators to the language does not increase expressive power.
Uncertainty-Aware Bayes' Rule and Its Applications

Shixiong Wang

2024-05-08

Abstract:Bayes' rule has enabled innumerable powerful algorithms of statistical signal processing and statistical machine learning. However, when there exist model misspecifications in prior distributions and/or data distributions, the direct application of Bayes' rule is questionable. Philosophically, the key is to balance the relative importance of prior and data distributions when calculating posterior distributions: if prior (resp. data) distributions are overly conservative, we should upweight the prior belief (resp. data evidence); if prior (resp. data) distributions are overly opportunistic, we should downweight the prior belief (resp. data evidence). This paper derives a generalized Bayes' rule, called uncertainty-aware Bayes' rule, to technically realize the above philosophy, i.e., to combat the model uncertainties in prior distributions and/or data distributions. Simulated and real-world experiments on classification and estimation showcase the superiority of the presented uncertainty-aware Bayes' rule over the conventional Bayes' rule: In particular, the uncertainty-aware Bayes classifier, the uncertainty-aware Kalman filter, the uncertainty-aware particle filter, and the uncertainty-aware interactive-multiple-model filter are suggested and validated.

Signal Processing,Methodology
An Information-Theoretic Foundation for the Weighted Updating Model

Jesse Aaron Zinn

DOI: https://doi.org/10.31235/osf.io/s4wfr

2016-07-21

Abstract:Weighted Updating generalizes Bayesian updating, allowing for biased beliefs by weighting the likelihood function and prior distribution with positive real exponents. I provide a rigorous foundation for the model by showing that transforming a distribution by exponential weighting (and normalizing) systematically affects the information entropy of the resulting distribution. For weights greater than one the resulting distribution has less information entropy than the original distribution, and vice versa. As the entropy of a distribution measures how informative a decision maker is treating the underlying observation(s), this result suggests a useful interpretation of the weights. For example, a weight greater than one on a likelihood function models an individual who is treating the associated observation(s) as being more informative than a perfect Bayesian would.
The expected demise of the Bayes factor

Christian P. Robert

DOI: https://doi.org/10.48550/arXiv.1506.08292

2015-07-27

Abstract:This note is a discussion commenting on the paper by Ly et al. on "Harold Jeffreys's Default Bayes Factor Hypothesis Tests: Explanation, Extension, and Application in Psychology" and on the perceived shortcomings of the classical Bayesian approach to testing, while reporting on an alternative approach advanced by Kamary, Mengersen, Robert and Rousseau (2014. <a class="link-https" data-arxiv-id="1412.2044" href="https://arxiv.org/abs/1412.2044">arXiv:1412.2044</a>) as a solution to this quintessential inference problem.

Methodology
'On Being Informed': Update Logics For Knowledge States

J Van Benthem

DOI: https://doi.org/10.1007/3-540-40006-0_1

2000-01-01

Abstract:Statements convey information, by modifying knowledge states of hearers and speakers. This dynamic aspect of communication goes beyond the usual role of logic as a provider of static ‘truth conditions’. But it can be modelled rather nicely in so-called ‘update logics’, which have been developed since the 1980s. These systems provide a fresh look at standard logic, letting the usual models undergo suitable changes as agents absorb the content of successive utterances or messages. This lecture is a brief Whig history of update logics, with an emphasis on many-agent epistemic languages. We discuss straight update, questions and answers, and the delightful complexities of communication under various constraints. We hope to convey the attraction of giving a dynamic twist to wellknown things, such as simple modal models, or basic epistemic formulas.
Mathematical theory of Bayesian statistics for unknown information source

Sumio Watanabe

DOI: https://doi.org/10.1098/rsta.2022.0151

2023-03-29

Abstract:In statistical inference, uncertainty is unknown and all models are wrong. That is to say, a person who makes a statistical model and a prior distribution is simultaneously aware that both are fictional candidates. To study such cases, statistical measures have been constructed, such as cross validation, information criteria and marginal likelihood; however, their mathematical properties have not yet been completely clarified when statistical models are under- or over-parametrized. We introduce a place of mathematical theory of Bayesian statistics for unknown uncertainty, which clarifies general properties of cross validation, information criteria and marginal likelihood, even if an unknown data-generating process is unrealizable by a model or even if the posterior distribution cannot be approximated by any normal distribution. Hence it gives a helpful standpoint for a person who cannot believe in any specific model and prior. This paper consists of three parts. The first is a new result, whereas the second and third are well-known previous results with new experiments. We show there exists a more precise estimator of the generalization loss than leave-one-out cross validation, there exists a more accurate approximation of marginal likelihood than Bayesian information criterion, and the optimal hyperparameters for generalization loss and marginal likelihood are different. This article is part of the theme issue ‘Bayesian inference: challenges, perspectives, and prospects’.

multidisciplinary sciences
A Good check on the Bayes factor

Nikola Sekulovski,Maarten Marsman,Eric-Jan Wagenmakers

DOI: https://doi.org/10.3758/s13428-024-02491-4

2024-09-04

Abstract:Bayes factor hypothesis testing provides a powerful framework for assessing the evidence in favor of competing hypotheses. To obtain Bayes factors, statisticians often require advanced, non-standard tools, making it important to confirm that the methodology is computationally sound. This paper seeks to validate Bayes factor calculations by applying two theorems attributed to Alan Turing and Jack Good. The procedure entails simulating data sets under two hypotheses, calculating Bayes factors, and assessing whether their expected values align with theoretical expectations. We illustrate this method with an ANOVA example and a network psychometrics application, demonstrating its efficacy in detecting calculation errors and confirming the computational correctness of the Bayes factor results. This structured validation approach aims to provide researchers with a tool to enhance the credibility of Bayes factor hypothesis testing, fostering more robust and trustworthy scientific inferences.
Generalized Bhattacharyya and Chernoff upper bounds on Bayes error using quasi-arithmetic means

Frank Nielsen

DOI: https://doi.org/10.1016/j.patrec.2014.01.002

2014-01-20

Abstract:Bayesian classification labels observations based on given prior information, namely class-a priori and class-conditional probabilities. Bayes' risk is the minimum expected classification cost that is achieved by the Bayes' test, the optimal decision rule. When no cost incurs for correct classification and unit cost is charged for misclassification, Bayes' test reduces to the maximum a posteriori decision rule, and Bayes risk simplifies to Bayes' error, the probability of error. Since calculating this probability of error is often intractable, several techniques have been devised to bound it with closed-form formula, introducing thereby measures of similarity and divergence between distributions like the Bhattacharyya coefficient and its associated Bhattacharyya distance. The Bhattacharyya upper bound can further be tightened using the Chernoff information that relies on the notion of best error exponent. In this paper, we first express Bayes' risk using the total variation distance on scaled distributions. We then elucidate and extend the Bhattacharyya and the Chernoff upper bound mechanisms using generalized weighted means. We provide as a byproduct novel notions of statistical divergences and affinity coefficients. We illustrate our technique by deriving new upper bounds for the univariate Cauchy and the multivariate $t$-distributions, and show experimentally that those bounds are not too distant to the computationally intractable Bayes' error.

Computer Vision and Pattern Recognition,Information Theory

Bayes = Blackwell, Almost

Information Validates the Prior: A Theorem on Bayesian Updating and Applications

Judicious Judgment Meets Unsettling Updating: Dilation, Sure Loss, and Simpson's Paradox

Conditional Probability Meets Update Logic.

Justifying the Norms of Inductive Inference

Updating information based on generalized credal sets. Part 1: Finite case

State Information in Bayesian Games

Dynamic Update with Probabilities

Updating information based on generalized credal sets. Part 2: General case

Almost-Bayesian Quadratic Persuasion (Extended Version)

"Not only defended but also applied": The perceived absurdity of Bayesian inference

Non-Bayesian Persuasion

Blackwell's Informativeness Ranking with Uncertainty-Averse Preferences

Conditional Belief, Knowledge and Probability

Uncertainty-Aware Bayes' Rule and Its Applications

An Information-Theoretic Foundation for the Weighted Updating Model

The expected demise of the Bayes factor

'On Being Informed': Update Logics For Knowledge States

Mathematical theory of Bayesian statistics for unknown information source

A Good check on the Bayes factor

Generalized Bhattacharyya and Chernoff upper bounds on Bayes error using quasi-arithmetic means