Abstract:This short text tried to establish a big picture of what evidential statistics is about and how an ideal inference method should behave. Moreover, by examining shortcomings of some of the currently used methods for measuring evidence and utilizing some intuitive principles, we motivated the Relative Belief Ratio as the primary method of characterizing statistical evidence. Number of topics has been omitted for the interest of this text and the reader is strongly advised to refer to (Evans, 2015) as the primary source for further readings of the subject.
What problem does this paper attempt to address?
The core problem that this paper attempts to solve is how to accurately measure and interpret statistical evidence in statistics. Specifically, the author explores the effectiveness and limitations of different methods in the process of statistical inference, especially for common methods such as likelihood inference, frequentist methods (such as p - values and confidence intervals), and Bayesian inference.
### Overview of the Main Problems in the Paper:
1. **Definition of Statistical Problems**:
- Starting from Hume's problem of induction, the author discusses probability theory as a framework for solving the problem of induction and points out that although probability theory provides a consistent reasoning system, it does not stipulate how to interpret the obtained probabilities.
- The fundamental problem in statistics is how to infer the true relative frequency function from the observed data subset. To this end, the author introduces some assumptions, for example, the data comes from a certain parameterized relative frequency function family \( M=\{f_{\theta}:\theta\in\Theta\} \).
2. **Falsifiability, Objectivity and Subjectivity**:
- The author emphasizes that subjective choices in statistical inference are inevitable, but these choices can be verified by objective information of assumptions.
- It is emphasized that scientific theories must be falsifiable (Popper, 2002), that is, each component of the statistical inference method must be able to be empirically tested.
3. **Randomness and Continuity**:
- The definition and characteristics of randomness are discussed, especially the concept of Kolmogorov complexity, and it is pointed out that randomness is difficult to be strictly defined and detected.
- The application of infinity and continuity in statistics and their potential problems, such as Borel's paradox, are explored.
4. **Decision Theory**:
- Two major schools of thought in statistics are distinguished: decision theory (the American school) and evidence theory (the British school). The author believes that some components in decision theory (such as loss functions) are difficult to verify and are therefore not considered in this paper.
5. **Methods for Measuring Statistical Evidence**:
- Several methods for measuring statistical evidence, such as pure likelihood inference, Birnbaum's theorem, frequentist methods (such as p - values and confidence intervals), and Bayesian inference, are discussed in detail.
- The advantages and limitations of each method are pointed out, especially several problems of p - values and confidence intervals, such as insensitivity to sample size and inability to effectively support the null hypothesis.
### Conclusion:
The paper aims to explore how to more effectively measure and interpret statistical evidence by comparing and analyzing different statistical inference methods. The author points out that the existing methods have their own advantages and disadvantages, and there are some common challenges, such as the handling of subjectivity, randomness and continuity. Ultimately, the author hopes to find a more logically rigorous and falsifiable statistical inference method to better serve scientific research.
### Examples of Formulas:
- Relative Frequency Function:
\[
f_X(x)=\frac{\#(\{\omega\in\Omega:X(\omega) = x\})}{\#(\Omega)}
\]
- Maximum Likelihood Estimation (MLE):
\[
\theta_{\text{MLE}}(x)=\arg\sup_{\theta}L(\theta|x)
\]
- p - value:
\[
p_{H_0}(x)=P_{H_0}(T(X)\geq T(x))
\]
Through these formulas and discussions, the author attempts to provide readers with a comprehensive understanding framework to help them better evaluate and apply statistical inference methods.