Abstract:<h3>Background</h3><p>Despite its popularity, issues concerning the estimation of power in multilevel logistic regression models are prevalent because of the complexity involved in its calculation (i.e., computer-simulation-based approaches). These issues are further compounded by the fact that the distribution of the predictors can play a role in the power to estimate these effects. To address both matters, we present a sample of cases documenting the influence that predictor distribution have on statistical power as well as a user-friendly, web-based application to conduct power analysis for multilevel logistic regression.</p><h3>Method</h3><p>Computer simulations are implemented to estimate statistical power in multilevel logistic regression with varying numbers of clusters, varying cluster sample sizes, and non-normal and non-symmetrical distributions of the Level 1/2 predictors. Power curves were simulated to see in what ways non-normal/unbalanced distributions of a binary predictor and a continuous predictor affect the detection of population effect sizes for main effects, a cross-level interaction and the variance of the random effects.</p><h3>Results</h3><p>Skewed continuous predictors and unbalanced binary ones require larger sample sizes at both levels than balanced binary predictors and normally-distributed continuous ones. In the most extreme case of imbalance (10% incidence) and skewness of a chi-square distribution with 1 degree of freedom, even 110 Level 2 units and 100 Level 1 units were not sufficient for all predictors to reach power of 80%, mostly hovering at around 50% with the exception of the skewed, continuous Level 2 predictor.</p><h3>Conclusions</h3><p>Given the complex interactive influence among sample sizes, effect sizes and predictor distribution characteristics, it seems unwarranted to make generic rule-of-thumb sample size recommendations for multilevel logistic regression, aside from the fact that larger sample sizes are required when the distributions of the predictors are not symmetric or balanced. The more skewed or imbalanced the predictor is, the larger the sample size requirements. To assist researchers in planning research studies, a user-friendly web application that conducts power analysis via computer simulations in the R programming language is provided. With this web application, users can conduct simulations, tailored to their study design, to estimate statistical power for multilevel logistic regression models.</p>

A more powerful test method for analyzing unreplicated factorial two-level experiments

A note on extending Scheffé's modified multiple-comparison procedure to other analysis situations

Rank-Based Procedures in Factorial Designs: Hypotheses about Nonparametric Treatment Effects

An Efficient Method for Identifying Clear Effects in Blocked Fractional Factorial Designs

Multivariate nonparametric methods in two-way balanced designs: performances and limitations in small samples

Simulation-Based Power-Analysis for Factorial ANOVA Designs

Simulation-Based Power Analysis for Factorial Analysis of Variance Designs

Statistical Inference and Power Analysis for Direct and Spillover Effects in Two-Stage Randomized Experiments

A Monte Carlo Study of the Effects of Common Method Variance on Significance Testing and Parameter Bias in Hierarchical Linear Modeling

Nonparametric Method for Clustered Data in Pre-Post Factorial Design

A Bayesian Perspective on the Analysis of Unreplicated Factorial Experiments Using Potential Outcomes.

Maxpro Designs for Experiments with Multiple Types of Branching and Nested Factors

Model-Selection-Based Approaches to Identifying the Optimal Number of Factors in Multilevel Exploratory Factor Analysis

Random-effect Based Test for Multinomial Logistic Regression: Choice of the Reference Level and Its Impact on the Testing

The relationship between statistical power and predictor distribution in multilevel logistic regression: a simulation-based approach

A New Multiple Testing Method in the Dependent Case

Power Enhancement in High Dimensional Cross-Sectional Tests

A powerful approach to the study of moderate effect modification in observational studies

A comparison of recent approaches to the analysis of repeated measurements

Equivalence Test in Multi-dimensional Space with Applications in A/B Testing

A factor-adjusted multiple testing of general alternatives