Abstract:A common feature of preclinical animal experiments is repeated measurement of the outcome, e.g., body weight measured in mice pups weekly for 20 weeks. Separate time point analysis or repeated measures analysis approaches can be used to analyze such data. Each approach requires assumptions about the underlying data and violations of these assumptions have implications for estimation of precision, and type I and type II error rates. Given the ethical responsibilities to maximize valid results obtained from animals used in research, our objective was to evaluate approaches to reporting repeated measures design used by investigators and to assess how assumptions about variation in the outcome over time impact type I and II error rates and precision of estimates. We assessed the reporting of repeated measures designs of 58 studies in preclinical animal experiments. We used simulation modelling to evaluate three approaches to statistical analysis of repeated measurement data. In particular, we assessed the impact of (a) repeated measure analysis assuming that the outcome had non-constant variation at all time points (heterogeneous variance) (b) repeated measure analysis assuming constant variation in the outcome (homogeneous variance), (c) separate ANOVA at individual time point in repeated measures designs. The evaluation of the three model fitting was based on comparing the p-values distributions, the type I and type II error rates and by implication, the shrinkage or inflation of standard error estimates from 1000 simulated dataset. Of 58 studies with repeated measures design, three provided a rationale for repeated measurement and 23 studies reported using a repeated-measures analysis approach. Of the 35 studies that did not use repeated-measures analysis, fourteen studies used only two time points to calculate weight change which potentially means collected data was not fully utilized. Other studies reported only select time points (n = 12) raising the issue of selective reporting. Simulation studies showed that an incorrect assumption about the variance structure resulted in modified error rates and precision estimates. The reporting of the validity of assumptions for repeated measurement data is very poor. The homogeneous variation assumption, which is often invalid for body weight measurements, should be confirmed prior to conducting the repeated-measures analysis using homogeneous covariance structure and adjusting the analysis using corrections or model specifications if this is not met.

Simulation methodologies to determine statistical power in laboratory animal research studies

Determining sample size adequacy for animal model studies in nutrition research: limits and ethical challenges of ordinary power calculation procedures

Using simulation studies to evaluate statistical methods

Effect size, sample size and power of forced swim test assays in mice: Guidelines for investigators to optimize reproducibility

Understanding p-values and significance

Enhancing Statistical Power While Maintaining Small Sample Sizes in Behavioral Neuroscience Experiments Evaluating Success Rates

Statistical simulations show that scientists need not increase overall sample size by default when including both sexes in in vivo studies

Recommendations to improve use and reporting of statistics in animal experiments

How much confidence do we need in animal experiments? Statistical assumptions in sample size estimation

Bayesian statistical concepts with examples from rodent toxicology studies

Simulation-Based Power Analyses for the Smallest Effect Size of Interest: A Confidence-Interval Approach for Minimum-Effect and Equivalence Testing

Type II error and statistical power in reports of small animal clinical trials.

Finding the right power balance: Better study design and collaboration can reduce dependence on statistical power

Protocol for a Systematic Review of Effect Sizes and Statistical Power in the Rodent Fear Conditioning Literature

Sample size determination and post hoc statistical power in bioequivalence studies

Reporting And Analysis Of Repeated Measurements In Preclinical Animals Experiments

Pitfalls and potentials in simulation studies: Questionable research practices in comparative simulation studies allow for spurious claims of superiority of any method

PDXpower: A Power Analysis Tool for Experimental Design in Pre-clinical Xenograft Studies for Uncensored and Censored Outcomes

Considering aspects of the 3Rs principles within experimental animal biology

Study design: think 'scientific value' not 'p-values'

Analysis of Bayesian posterior significance and effect size indices for the two-sample t-test to support reproducible medical research