Abstract:Estimating the probability of the binomial distribution is a basic problem, which appears in almost all introductory statistics courses and is performed frequently in various studies. In some cases, the parameter of interest is a difference between two probabilities, and the current work studies the construction of confidence intervals for this parameter when the sample size is small. Our goal is to find the shortest confidence intervals under the constraint of coverage probability being at least as large as a predetermined level. For the two-sample case, there is no known algorithm that achieves this goal, but different heuristics procedures have been suggested, and the present work aims at finding optimal confidence intervals. In the one-sample case, there is a known algorithm that finds optimal confidence intervals presented by Blyth and Still (J Am Stat Assoc 78(381):108–116, 1983). It is based on solving small and local optimization problems and then using an inversion step to find the global optimum solution. We show that this approach fails in the two-sample case and therefore, in order to find optimal confidence intervals, one needs to solve a global optimization problem, rather than small and local ones, which is computationally much harder. We present and discuss the suitable global optimization problem. Using the Gurobi package we find near-optimal solutions when the sample sizes are smaller than 15, and we compare these solutions to some existing methods, both approximate and exact. We find that the improvement in terms of lengths with respect to the best competitor varies between 1.5 and 5% for different parameters of the problem. Therefore, we recommend the use of the new confidence intervals when both sample sizes are smaller than 15. Tables of the confidence intervals are given in the Excel file in this link (https://technionmail-my.sharepoint.com/:f:/g/personal/ap_campus_technion_ac_il/El-213Kms51BhQxR8MmQJCYBDfIsvtrK9mQIey1sZnZWIQ?e=hxGunl).

Construction and Comparisons of Simultaneous Confidence Intervals for the Mean Difference of Multivariate Normal Distributions

An optimal exact confidence interval for the difference of two independent binomial proportions

Simultaneous Confidence Bands for Comparing Variance Functions of Two Samples Based on Deterministic Designs

Hessian and increasing-Hessian orderings of scale-shape mixtures of multivariate skew-normal distributions and applications

Two-Sample Inference in Highly Dispersed Negative Binomial Models

Simultaneous confidence intervals for pairwise multiple comparisons in a two-way unbalanced design with unequal variances

Multiple Comparisons of Several Log-normal Means under Heteroscedasticity

Cost-Efficient Fixed-Width Confidence Intervals for the Difference of Two Bernoulli Proportions

Simultaneous confidence interval construction for many-to-one of proportion ratios of bilateral correlated data

A Practical Approach for Comparing Means of Two Groups Without Equal Variance Assumption

Statistical Inference for the Difference Between the Best Treatment Mean and a Control Mean

Simultaneous Confidence Intervals for the Difference of Two Survival Functions

Simultaneous Confidence Intervals for Several Inverse Gaussian Populations

Comparing Ratios of the Mean to a Power of Variance in Two Samples Via Self-Normalized Test Statistics

Comparing Confidence Intervals for the Mean of Symmetric and Skewed Distributions

Simultaneous Confidence Band for the Difference of Regression Functions of Two Samples

One-tailed asymptotic inferences for the difference of proportions: Analysis of 97 methods of inference

Optimal confidence interval for the difference between proportions

On Testing the Mean Equivalence of Treatments from Correlated Normal Populations

A Multiple Comparisons Procedure for Detecting Differences Between Treatments and A Control in Two-Factor Experiments

Max–min Multiple Comparison Procedure for Isotonic Dose–response Curves