Abstract:BackgroundSmall number of clusters and large variation of cluster sizes commonly exist in cluster-randomized trials (CRTs) and are often the critical factors affecting the validity and efficiency of statistical analyses. F tests are commonly used in the generalized linear mixed model (GLMM) to test intervention effects in CRTs. The most challenging issue for the approximate Wald F test is the estimation of the denominator degrees of freedom (DDF). Some DDF approximation methods have been proposed, but their small sample performances in analysing binary outcomes in CRTs with few heterogeneous clusters are not well studied.MethodsThe small sample performances of five DDF approximations for the F test are compared and contrasted under CRT frameworks with simulations. Specifically, we illustrate how the intraclass correlation (ICC), sample size, and the variation of cluster sizes affect the type I error and statistical power when different DDF approximation methods in GLMM are used to test intervention effect in CRTs with binary outcomes. The results are also illustrated using a real CRT dataset.ResultsOur simulation results suggest that the Between-Within method maintains the nominal type I error rates even when the total number of clusters is as low as 10 and is robust to the variation of the cluster sizes. The Residual and Containment methods have inflated type I error rates when the cluster number is small (<30) and the inflation becomes more severe with increased variation in cluster sizes. In contrast, the Satterthwaite and Kenward-Roger methods can provide tests with very conservative type I error rates when the total cluster number is small (<30) and the conservativeness becomes more severe as variation in cluster sizes increases. Our simulations also suggest that the Between-Within method is statistically more powerful than the Satterthwaite or Kenward-Roger method in analysing CRTs with heterogeneous cluster sizes, especially when the cluster number is small.ConclusionWe conclude that the Between-Within denominator degrees of freedom approximation method for F tests should be recommended when the GLMM is used in analysing CRTs with binary outcomes and few heterogeneous clusters, due to its type I error properties and relatively higher power.

An evaluation of computational methods for aggregate data meta-analyses of diagnostic test accuracy studies

Laplace approximation, penalized quasi-likelihood, and adaptive Gauss–Hermite quadrature for generalized linear mixed models: towards meta-analysis of binary outcome with sparse data

Joint meta-analysis of two diagnostic tests accounting for within and between studies dependence

A multinomial quadrivariate D-vine copula mixed model for meta-analysis of diagnostic studies in the presence of non-evaluable subjects

Efficient Integration of Aggregate Data and Individual Patient Data in One-Way Mixed Models

A re-analysis of about 60,000 sparse data meta-analyses suggests that using an adequate method for pooling matters

Decision Curve Analysis: a Technical Note

Meta‐Analysis of Diagnostic Accuracy Studies With Multiple Thresholds: Comparison of Approaches in a Simulation Study

A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes

An extended trivariate vine copula mixed model for meta-analysis of diagnostic studies in the presence of non-evaluable outcomes

Aggregating multiple test results to improve medical decision-making

Testing for Publication Bias in Diagnostic Meta-Analysis: A Simulation Study

Simulation models for aggregated data meta-analysis: Evaluation of pooling effect sizes and publication biases

A Regularization-Based Adaptive Test for High-Dimensional Generalized Linear Models

Comparing denominator degrees of freedom approximations for the generalized linear mixed model in analyzing binary outcome in small sample cluster-randomized trials

An one-factor copula mixed model for joint meta-analysis of multiple diagnostic tests

Hierarchical multinomial processing tree models for meta-analysis of diagnostic accuracy studies

Empirical Comparisons of 12 Meta-analysis Methods for Synthesizing Proportions of Binary Outcomes

Assessing method agreement for paired repeated binary measurements administered by multiple raters

Meta-analysis of dichotomous and ordinal tests without a gold standard

A comprehensive analysis comparing linear and generalized linear models in detecting adaptive SNPs