Abstract:Every philosophy has holes, and it is the responsibility of proponents of a philosophy to point out these problems. Here are a few holes in Bayesian data analysis: (1) the usual rules of conditional probability fail in the quantum realm, (2) flat or weak priors lead to terrible inferences about things we care about, (3) subjective priors are incoherent, (4) Bayesian decision picks the wrong model, (5) Bayes factors fail in the presence of flat or weak priors, (6) for Cantorian reasons we need to check our models, but this destroys the coherence of Bayesian inference. Some of the problems of Bayesian statistics arise from people trying to do things they shouldn't be trying to do, but other holes are not so easily patched. In particular, it may be a good idea to avoid flat, weak, or conventional priors, but such advice, if followed, would go against the vast majority of Bayesian practice and requires us to confront the fundamental incoherence of Bayesian inference. This does not mean that we think Bayesian inference is a bad idea, but it does mean that there is a tension between Bayesian logic and Bayesian workflow which we believe can only be resolved by considering Bayesian logic as a tool, a way of revealing inevitable misfits and incoherences in our model assumptions, rather than as an end in itself.
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is to explore several theoretical and practical problems in Bayesian statistics. Specifically, the author points out several "loopholes" in Bayesian data analysis, and these problems include but are not limited to:
1. **Failure of the conditional probability rule in quantum physics**:
In quantum physics, the usual conditional probability rule does not apply. For example, in the double - slit experiment, when both slits are open, the position distribution \(p_3(y)\) of photons passing through the screen cannot be simply represented as the mixed distribution \(p_4(y) = 0.5p_1(y)+ 0.5p_2(y)\) when the two slits are open separately. This violates the basic rule of Bayesian statistics, that is, updating the probability distribution through conditioning.
2. **Unreasonable inferences caused by weak or flat priors**:
Using weak or flat priors may lead to unreasonable inferences about the parameters of interest. For example, when the estimated value \(\hat{\theta}\) is equal to the standard error \(s\), the posterior probability \(Pr(\theta > 0|y)=0.84\), which means that there is approximately a 5/6 probability of believing that the effect is positive. Such a conclusion is too strong in practice, especially when the data itself may be noisy.
3. **Inconsistency of subjective priors**:
Subjective priors are sometimes considered unquestionable because they represent personal beliefs. However, if a person can determine an indisputable subjective prior before seeing the data, then he should be able to directly derive the posterior distribution from the data without the need for formal Bayesian statistical methods. Therefore, the setting of subjective priors is actually not reasonable.
4. **Complexity of Bayesian decision theory**:
Bayesian decision theory may have problems in some cases, especially when evaluating models. For example, when using the Bayesian factor to compare models, the integral \(\int p(\theta|M)p(y|\theta,M)d\theta\) is very sensitive to certain aspects of the prior distribution \(p(\theta|M)\), even if these aspects have little impact on the conditional posterior distribution.
5. **Cantor's corner**:
Models are continuously improved and extended until they can no longer adapt to new problems. This process is similar to Cantor's diagonal argument, indicating that we can never fully foresee the changes required for future models. Therefore, direct Bayesian inference (such as discrete model averaging or continuous model averaging) is essentially impossible because it requires foreseeing future model changes.
### Summary
The core purpose of the paper is to reveal some basic problems in Bayesian statistics and discuss the impact of these loopholes on practical applications. The author emphasizes that although Bayesian reasoning is a powerful tool, there is a tension between its logic and work flow, and Bayesian logic needs to be regarded as a tool to reveal the inevitable inconsistencies and mismatches in model assumptions, rather than as the ultimate goal.
Through these discussions, the author hopes to deepen the understanding of Bayesian reasoning and propose some potential solutions to address these theoretical and practical challenges.