Abstract:Structural Equation Models (SEMs) are routinely used in the analysis of empirical data by researchers from different scientific fields such as psychologists or economists. In some fields, such as in ecology, SEMs have only started recently to attract attention and thanks to dedicated software packages the use of SEMs has steadily increased. Yet, common analysis practices in such fields that might be transposed from other statistical techniques such as model acceptance or rejection based on p-value screening might be poorly fitted for SEMs especially when these models are used to confirm or reject hypotheses. In this simulation study, SEMs were fitted via two commonly used R packages: lavaan and piecewiseSEM. Five different data-generation scenarios were explored: (i) random, (ii) exact, (iii) shuffled, (iv) underspecified and (v) overspecified. In addition, sample size and model complexity were also varied to explore their impact on various global and local model fitness indices. The results showed that not one single model index should be used to decide on model fitness but rather a combination of different model fitness indices is needed. The global chi-square test for lavaan or the Fisher's C statistic for piecewiseSEM were, in isolation, poor indicators of model fitness. In addition, the simulations showed that to achieve sufficient power to detect individual effects, adequate sample sizes are required. Finally, BIC showed good capacity to select models closer to the truth especially for more complex models. I provide, based on these results, a flowchart indicating how information from different metrics may be combined to reveal model strength and weaknesses. Researchers in scientific fields with little experience in SEMs, such as in ecology, should consider and accept these limitations.

Power and Sample Size for the Root Mean Square Error of Approximation Test of not Close Fit in Structural Equation Modeling

Evaluating Small Sample Approaches for Model Test Statistics in Structural Equation Modeling

Power Analysis for Parameter Estimation in Structural Equation Modeling: A Discussion and Tutorial

Model-implied simulation-based power estimation for correctly specified and distributionally misspecified models: Applications to nonlinear and linear structural equation models

Small Sample Solutions for Structural Equation Modeling

SAMPLE SIZE CALCULATION IN STRUCTURAL EQUATION MODELING OF EQUILIBRIUM

Estimating statistical power for structural equation models in developmental cognitive science: A tutorial in R

Power analyses for measurement model misspecification and response shift detection with structural equation modeling

Methods and Rule-of-Thumbs in The Determination of Minimum Sample Size When Appling Structural Equation Modelling: A Review

Evaluating Structural Equation Models with Unobservable Variables and Measurement Error

Equivalence Testing Based Fit Index: Standardized Root Mean Squared Residual

Minimizing Sample Size When Using Exploratory Factor Analysis for Measurement

On Power and Sample Size Computation for Multiple Testing Procedures

Sample Size Estimation for Random-effects Models: Balancing Precision and Feasibility in Panel Studies

How robust are Structural Equation Models to model miss-specification? A simulation study

Power and sample size calculations. A review and computer program

“How Well Does Your Structural Equation Model Fit Your Data?”: Is Marcoulides and Yuan’s Equivalence Test the Answer?

Performance of Model Fit and Selection Indices for Bayesian Structural Equation Modeling with Missing Data

Impact of Sample Size and Variability on the Power and Type I Error Rates of Equivalence Tests: A Simulation Study.

The Extended Anderson and Hauck Tests and Sample Size Procedures for Equivalence Assessment in Simple Linear Regressions

Sample Size Planning for Detecting Mediation Effects: A Power Analysis Procedure Considering Uncertainty in Effect Size Estimates