Abstract:When applied to frequency tables with small expected cell counts, Pearson chi-squared test statistics may be asymptotically inconsistent even in cases in which a satisfactory chi-squared approximation exists for the distribution under the null hypothesis. This problem is particularly important in cases in which the number of cells is large and the expected cell counts are quite variable. To illustrate this bias of the chi-squared test, this article considers the Pearson chi-squared test of the hypothesis that the cell probabilities for a multinomial frequency table have specified values. In this case, the expected value and variance of the Pearson chi-square may be evaluated under both the null and alternative hypotheses. When the number of cells is large, normal approximations and discrete Edgeworth expansions may also be used to assess the size and power of the Pearson chi-squared test. These analyses show that unless all cell probabilities are equal, it is possible to select a significance level and cell probabilities under the alternative hypothesis such that the power is less than the size of the test. As shown by exact calculations, the difference may be substantial even in cases in which all expected cell sizes are at least 5 under the null hypothesis. The use of moments shows that given any minimum expected cell size under the null hypothesis and given any significance level, it is possible to make the power arbitrarily close to 0 by the selection of a large enough number of cells in the table and suitable cell probabilities for the null and alternative hypotheses. The normal approximations for the distribution of the Pearson chi-squared statistic permit the size of this bias to be assessed in less-extreme cases involving tables with many cells. These results imply that caution must be exercised in the application of Pearson chi-squared statistics to sparse contingency tables with many cells. An alternative to the Pearson chi-square, proposed by Zelterman (1986), avoids some of the problems. Exact calculation, however, shows that the alternative statistic does not eliminate all problems of bias. The problems described in this article clearly extend to more general applications of the Pearson chi-squared statistic.

Exact Chi-Square and Fisher's Exact Probability Test for 3 by 2 Cross-Classification Tables

Fisher's Exact Probability Test for Cross-Classification Tables

Subroutines for Computing Exact CHI-Square and Fisher's Exact Probability Tests

A Computer Program for Fisher's Exact Probability Test

A Program for Computing Fisher's Exact Probability Test and the Coefficient of Association λ for n × m Contingency Tables

A Rapid FORTRAN Subroutine for the Fisher Exact Probability Test

A Program for Calculating the Exact Probability Along with Explorations of M By N Contingency Tables

Applicability of chi-square to 2???2 contingency tables with small expected cell frequencies.

Exact p-values for pairwise comparison of Friedman rank sums, with application to comparing classifiers

A Simple Algorithm for Exact Multinomial Tests

Likelihood Ratio Test and the Evidential Approach for 2 × 2 Tables

Continuity Correction for Fisher's Exact Probability Test

A Warning on the Use of Chi-Squared Statistics with Frequency Tables with Small Expected Cell Counts

Comparing Evolutionary Rates Using An Exact Test for 2x2 Tables with Continuous Cell Entries

The Classification Permutation Test: A Nonparametric Test for Equality of Multivariate Distributions

Simultaneous Analysis of Variance Tests Using Friedman's χ r2 Statistics

On an Exact and Nonparametric Test for the Separability of Two Classes by Means of a Simple Threshold

Differentially Private Chi-Squared Hypothesis Testing: Goodness of Fit and Independence Testing

Optimal exact tests for composite alternative hypotheses on cross tabulated data

Two-Sample Test Based on Classification Probability

Test for symmetry in $2 \times 2$ contingency tables with nonignorable nonresponses