Abstract:Confidence interval performance is typically assessed in terms of two criteria: coverage probability and interval width (or margin of error). In this article, we assess the performance of four common proportion interval estimators: the Wald, Clopper-Pearson (exact), Wilson and Agresti-Coull, in the context of rare-event probabilities. We define the interval precision in terms of a relative margin of error which ensures consistency with the magnitude of the proportion. Thus, confidence interval estimators are assessed in terms of achieving a desired coverage probability whilst simultaneously satisfying the specified relative margin of error. We illustrate the importance of considering both coverage probability and relative margin of error when estimating rare-event proportions, and show that within this framework, all four interval estimators perform somewhat similarly for a given sample size and confidence level. We identify relative margin of error values that result in satisfactory coverage while being conservative in terms of sample size requirements, and hence suggest a range of values that can be adopted in practice. The proposed relative margin of error scheme is evaluated analytically, by simulation, and by application to a number of recent studies from the literature.
What problem does this paper attempt to address?
This paper attempts to solve the problem of constructing binomial proportion confidence intervals (CIs) when dealing with the probabilities of rare events. Specifically, the author focuses on how to define the relative margin of error consistent with the proportion size to ensure that when estimating small proportions (such as the adverse reaction rate in clinical trials, the defect rate in manufacturing, etc.), the performance of the confidence interval can both reach the required coverage probability and meet specific relative error requirements.
### Background of the Paper
In applied statistics, a fundamental problem is to construct a confidence interval for the binomial proportion \( p \). In many application scenarios, events are rare in large populations, for example:
- In clinical statistics, \( p \) may represent the proportion of patients with treatment side effects.
- In manufacturing, the number of defective parts is very small compared to the large number of parts produced.
- In the aviation industry, strict regulations ensure that safety events are regarded as rare events.
### Importance of the Problem
Determining the order of magnitude of \( p \) in "large populations" is very important. For example, in high - throughput manufacturing, differences in the order of magnitude of the failure rate have a significant impact on the number of defects and/or product returns. Therefore, for small proportion \( p \), more specific sample - size guidance is required, which is not fully covered in the existing literature.
### Shortcomings of Existing Research
Existing research on constructing confidence intervals mainly focuses on cases where the proportion \( p \) is large, while there is less research on small proportion \( p \). In addition, few literatures discuss the importance of the relative margin of error in the context of small proportion \( p \). The relative margin of error is especially important for rare events because the margin of error should change with the size of the proportion \( p \).
### Main Contributions of the Paper
1. **Introducing the Relative Margin of Error**: The author proposes to use the relative margin of error \( \epsilon_R=\frac{\epsilon}{p} \) to evaluate the performance of the confidence interval, ensuring that the margin of error is consistent with the size of the proportion \( p \).
2. **Evaluating Four Common Confidence Interval Estimators**: Wald, Clopper - Pearson (exact), Wilson and Agresti - Coull. These estimators perform similarly given the sample size and confidence level.
3. **Providing Practical Suggestions**: The author suggests that in the context of small proportion \( p \), a relative margin of error \( \epsilon_R\in[0.1, 0.5] \) is a reasonable solution, which can ensure the coverage probability while avoiding excessive sample size requirements.
### Conclusion
By considering the relative margin of error, the author shows that in the context of small proportion \( p \), the four commonly - used confidence interval estimators perform similarly in many cases. Although the Wilson interval performs best in some aspects, all four methods can provide reasonable estimation accuracy and coverage probability within an appropriate relative margin of error. These results provide valuable guidance for practical applications, especially in cases where accurate estimation of the probabilities of rare events is required.