Abstract:The premise in experiments with repeated measures is that observations taken in the same experimental unit are correlated and that correlations decrease proportionally to the increase in the distance between measurements in time or space. Nevertheless, these experiments are often analyzed as if the correlations between the repeated measures were constant or using methods that only consider correlations different, which may impact on the rejection rate of the null hypothesis, and ultimately type I error rate and statistical power. In this context, this study investigated the application of mixed linear models with different assumptions about the covariance matrix in data sets from simulated experiments with repeated measures. 84 scenarios that varied in terms of the covariance matrix pattern (14 structures), number of repeated measurements (4 and 8) and sample size (4, 8 and 12) were evaluated. 10,000 data sets were simulated for each scenario based on a multivariate normal distribution and were subsequently analyzed using mixed linear models. Type I error rate and statistical power for the hypothesis test of the interaction between treatments and repeated measures were estimated as the proportion of p values less than or equal to 0.01 or 0.05 out of a total of 10,000 tests for each scenario. The models were also evaluated for their ability to fit the data using Bayesian Information Criteria (BIC). Thus, the frequency with which the covariance structures were chosen by the selection criteria was computed. Results indicate that the assumption chosen most frequently by the information criteria resulted from the specified covariance structure that corresponded to the empirical covariance structure of the analyzed data sets, particularly for those with larger number of repeated measures and sample sizes. Results also indicate that the use of covariance models that do not recognize heterogeneous correlations between repeated measures can inflate type I error or reduce it to very conservative levels, which may affect the conclusion of agricultural experiments. For a 5% significance level, type I error bias was greater than 2α, while for 1% significance level, bias was over 5α. In addition, the statistical power was reduced when the assumption about the covariance matrix of the data sets did not correspond to the empirical covariance structure, particularly for those data sets with a smaller sample size.

Improvement of the GFI and AGFI indexes in structural equation models by the covariance between repetitions in the observed variables

Parametric Models of Covariance Matrices for Repeated Measures: A Simulation Study on Fit, Error and Statistical Power in Mixed Linear Models

Cutoff criteria for overall model fit indexes in generalized structured component analysis

Adapting fit indices for Bayesian structural equation modeling: Comparison to maximum likelihood.

Equivalence Testing Based Fit Index: Standardized Root Mean Squared Residual

Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance

Evaluating Small Sample Approaches for Model Test Statistics in Structural Equation Modeling

Effect of the Number of Variables on Measures of Fit in Structural Equation Modeling

Model-Selection-Based Approaches to Identifying the Optimal Number of Factors in Multilevel Exploratory Factor Analysis

Modification of Factor Analysis Models in Covariance Structure Analysis a Monte Carlo Study

Optimal Number of Replications for Obtaining Stable Dynamic Fit Index Cutoffs

“How Well Does Your Structural Equation Model Fit Your Data?”: Is Marcoulides and Yuan’s Equivalence Test the Answer?

Goodness of fit of relational event models

Fit Indices in Structural Equation Modeling and Confirmatory Factor Analysis: Reporting Guidelines

Performance of Model Fit and Selection Indices for Bayesian Structural Equation Modeling with Missing Data

A goodness of fit framework for relational event models

Structural Equation Models in a Redundancy Analysis Framework With Covariates

Evaluating Structural Equation Models with Unobservable Variables and Measurement Error

A Simulation Study of Polychoric Instrumental Variable Estimation in Structural Equation Models

Structural Equation Modeling: Threshold Criteria for Assessing Model Fit

The Effect of Latent and Error Non-Normality on Measures of Fit in Structural Equation Modeling