Abstract:Estimating the probability of the binomial distribution is a basic problem, which appears in almost all introductory statistics courses and is performed frequently in various studies. In some cases, the parameter of interest is a difference between two probabilities, and the current work studies the construction of confidence intervals for this parameter when the sample size is small. Our goal is to find the shortest confidence intervals under the constraint of coverage probability being at least as large as a predetermined level. For the two-sample case, there is no known algorithm that achieves this goal, but different heuristics procedures have been suggested, and the present work aims at finding optimal confidence intervals. In the one-sample case, there is a known algorithm that finds optimal confidence intervals presented by Blyth and Still (J Am Stat Assoc 78(381):108–116, 1983). It is based on solving small and local optimization problems and then using an inversion step to find the global optimum solution. We show that this approach fails in the two-sample case and therefore, in order to find optimal confidence intervals, one needs to solve a global optimization problem, rather than small and local ones, which is computationally much harder. We present and discuss the suitable global optimization problem. Using the Gurobi package we find near-optimal solutions when the sample sizes are smaller than 15, and we compare these solutions to some existing methods, both approximate and exact. We find that the improvement in terms of lengths with respect to the best competitor varies between 1.5 and 5% for different parameters of the problem. Therefore, we recommend the use of the new confidence intervals when both sample sizes are smaller than 15. Tables of the confidence intervals are given in the Excel file in this link (https://technionmail-my.sharepoint.com/:f:/g/personal/ap_campus_technion_ac_il/El-213Kms51BhQxR8MmQJCYBDfIsvtrK9mQIey1sZnZWIQ?e=hxGunl).

Estimation of Gini Index within Pre-Specied Error Bound

Designing of Gini-chart for Exponential, T, Logistic and Laplace Distributions.

Gini estimation under infinite variance

How to (Not) Estimate Gini Coefficients for Fat Tailed Variables

On the near-optimality of betting confidence sets for bounded means

Calculation of sample size guaranteeing the required width of the empirical confidence interval with predefined probability

Efficient Concentration with Gaussian Approximation

Nonparametric Gini-Frisch bounds

Estimation of the regression slope by means of Gini's cograduation index

Using the Gini coefficient to characterize the shape of computational chemistry error distributions

Binomial Confidence Intervals for Rare Events: Importance of Defining Margin of Error Relative to Magnitude of Proportion

Robust Estimation of Inequality from Binned Incomes

Optimal confidence interval for the difference between proportions

Estimating means of bounded random variables by betting

Gini index estimation for lifetime data

Concentration-based confidence intervals for U-statistics

New multivariate Gini's indices

Coupling the Gini and Angles to Evaluate Economic Dispersion

Gini index based goodness-of-fit test for the Lindley distribution

Two more ways of spelling Gini Coefficient with Applications

Extreme and Inference for Tail Gini Functionals with Applications in Tail Risk Measurement