Abstract:Estimating the probability of the binomial distribution is a basic problem, which appears in almost all introductory statistics courses and is performed frequently in various studies. In some cases, the parameter of interest is a difference between two probabilities, and the current work studies the construction of confidence intervals for this parameter when the sample size is small. Our goal is to find the shortest confidence intervals under the constraint of coverage probability being at least as large as a predetermined level. For the two-sample case, there is no known algorithm that achieves this goal, but different heuristics procedures have been suggested, and the present work aims at finding optimal confidence intervals. In the one-sample case, there is a known algorithm that finds optimal confidence intervals presented by Blyth and Still (J Am Stat Assoc 78(381):108–116, 1983). It is based on solving small and local optimization problems and then using an inversion step to find the global optimum solution. We show that this approach fails in the two-sample case and therefore, in order to find optimal confidence intervals, one needs to solve a global optimization problem, rather than small and local ones, which is computationally much harder. We present and discuss the suitable global optimization problem. Using the Gurobi package we find near-optimal solutions when the sample sizes are smaller than 15, and we compare these solutions to some existing methods, both approximate and exact. We find that the improvement in terms of lengths with respect to the best competitor varies between 1.5 and 5% for different parameters of the problem. Therefore, we recommend the use of the new confidence intervals when both sample sizes are smaller than 15. Tables of the confidence intervals are given in the Excel file in this link (https://technionmail-my.sharepoint.com/:f:/g/personal/ap_campus_technion_ac_il/El-213Kms51BhQxR8MmQJCYBDfIsvtrK9mQIey1sZnZWIQ?e=hxGunl).

Searching the differences through the tails of distributions using an approach based on Mahalanobis distance and percentile bootstrap

Non-parametric Tests for the Tail Equivalence Via Empirical Likelihood

A robust test approach for equality of mean vectors of two independent groups under the multivariate Behrens-Fisher problem

One-tailed asymptotic inferences for the difference of proportions: Analysis of 97 methods of inference

Examining Tail Index Estimators in New Pareto Distribution: Monte Carlo Simulations and Income Data Applications

Multiple Comparisons of Treatment Against Control under Unequal Variances Using Parametric Bootstrap

Empirical Likelihood Based Tests for Detecting the Presence of Significant Predictors in Marginal Quantile Regression

Comparing Ratios of the Mean to a Power of Variance in Two Samples Via Self-Normalized Test Statistics

A tail adaptive approach for change point detection.

Robustness and accuracy of methods for high dimensional data analysis based on Student's t statistic

A bootstrap semiparametric homogeneity test for the distributions of multigroup proportional data, with applications to analysis of quality of life outcomes in clinical trials

Mistaken identities lead to missed opportunities: Testing for mean differences in partially matched data

Optimal confidence interval for the difference between proportions

Parametric Bootstrap Confidence Interval Estimation for the Percentile and Difference between the Percentiles of Delta-Lognormal Distributions with Application to Rainfall Data in Thailand

The power of visualizing distributional differences: formal graphical n-sample tests

A Parametric Bootstrap Approach for One-Way ANOVA Under Unequal Variances with Unbalanced Data.

The New and Improved Two-Sample t Test

Cost-Efficient Fixed-Width Confidence Intervals for the Difference of Two Bernoulli Proportions

A distance based two-sample test of means difference for multivariate datasets

Analysis of small sample size studies using nonparametric bootstrap test with pooled resampling method

Comparison of Different Estimation Methods for The Inverse Power Perk distribution with Applications in Engineering and ActuarialSciences