Abstract:BackgroundSmall number of clusters and large variation of cluster sizes commonly exist in cluster-randomized trials (CRTs) and are often the critical factors affecting the validity and efficiency of statistical analyses. F tests are commonly used in the generalized linear mixed model (GLMM) to test intervention effects in CRTs. The most challenging issue for the approximate Wald F test is the estimation of the denominator degrees of freedom (DDF). Some DDF approximation methods have been proposed, but their small sample performances in analysing binary outcomes in CRTs with few heterogeneous clusters are not well studied.MethodsThe small sample performances of five DDF approximations for the F test are compared and contrasted under CRT frameworks with simulations. Specifically, we illustrate how the intraclass correlation (ICC), sample size, and the variation of cluster sizes affect the type I error and statistical power when different DDF approximation methods in GLMM are used to test intervention effect in CRTs with binary outcomes. The results are also illustrated using a real CRT dataset.ResultsOur simulation results suggest that the Between-Within method maintains the nominal type I error rates even when the total number of clusters is as low as 10 and is robust to the variation of the cluster sizes. The Residual and Containment methods have inflated type I error rates when the cluster number is small (<30) and the inflation becomes more severe with increased variation in cluster sizes. In contrast, the Satterthwaite and Kenward-Roger methods can provide tests with very conservative type I error rates when the total cluster number is small (<30) and the conservativeness becomes more severe as variation in cluster sizes increases. Our simulations also suggest that the Between-Within method is statistically more powerful than the Satterthwaite or Kenward-Roger method in analysing CRTs with heterogeneous cluster sizes, especially when the cluster number is small.ConclusionWe conclude that the Between-Within denominator degrees of freedom approximation method for F tests should be recommended when the GLMM is used in analysing CRTs with binary outcomes and few heterogeneous clusters, due to its type I error properties and relatively higher power.

Comparison of Statistical Methods for Analysis of Threshold Model of Two-Stage Clustered Randomized Data

Comparison of Different Methods for Analyzing Data Obtained from Stratified Cluster Random Sampling

Modified Poisson Regression Model for Data of Prospective Studies with Common Outcomes

Comparison of tests for categorical data from a stratified cluster randomized trial

Model-based Inference for Two-Stage Cluster Samples Subject to Nonignorable Item Nonresponse

A comparison of methods for the analysis of binomial proportion data in behavioral research

Comparison of statistical methods for the analysis of patient-reported outcomes in randomised controlled trials: A simulation study

Model-assisted analyses of cluster-randomized experiments

A novel statistical method for comparing effectiveness of two treatments-simulated randomized controlled trials

Statistical Considerations for Evaluating Treatment Effect under Various Non-proportional Hazard Scenarios

Cluster Randomized Trials with a Pretest and Posttest: Equivalence of Three-, Two- and One-Level Analyses, and Sample Size Calculation

Exploring Statistical Methods in Clinical Trials When Some of the Patients are Treated against the Randomization

Intent-to-treat Analysis of Cluster Randomized Trials when Clusters Report Unidentifiable Outcome Proportions.

Comparing cluster-level dynamic treatment regimens using sequential, multiple assignment, randomized trials: Regression estimation and sample size considerations

Marginal Regression on Transient State Occupation Probabilities with Clustered Multistate Process Data

Comparison of Dichotomized and Distributional Approaches in Rare Event Clinical Trial Design: a Fixed Bayesian Design

A comparison of four methods for the analysis of N-of-1 trials

The Evaluation of Some Two-stage Sampling Methods

Comparing denominator degrees of freedom approximations for the generalized linear mixed model in analyzing binary outcome in small sample cluster-randomized trials

A Two-Stage Approach to Differentiating Normal and Aberrant Behavior in Computer Based Testing

Latent Class Modeling with A Time-To-Event Distal Outcome: A Comparison of One, Two and Three-Step Approaches