Abstract:Structural Equation Models (SEMs) are routinely used in the analysis of empirical data by researchers from different scientific fields such as psychologists or economists. In some fields, such as in ecology, SEMs have only started recently to attract attention and thanks to dedicated software packages the use of SEMs has steadily increased. Yet, common analysis practices in such fields that might be transposed from other statistical techniques such as model acceptance or rejection based on p-value screening might be poorly fitted for SEMs especially when these models are used to confirm or reject hypotheses. In this simulation study, SEMs were fitted via two commonly used R packages: lavaan and piecewiseSEM. Five different data-generation scenarios were explored: (i) random, (ii) exact, (iii) shuffled, (iv) underspecified and (v) overspecified. In addition, sample size and model complexity were also varied to explore their impact on various global and local model fitness indices. The results showed that not one single model index should be used to decide on model fitness but rather a combination of different model fitness indices is needed. The global chi-square test for lavaan or the Fisher's C statistic for piecewiseSEM were, in isolation, poor indicators of model fitness. In addition, the simulations showed that to achieve sufficient power to detect individual effects, adequate sample sizes are required. Finally, BIC showed good capacity to select models closer to the truth especially for more complex models. I provide, based on these results, a flowchart indicating how information from different metrics may be combined to reveal model strength and weaknesses. Researchers in scientific fields with little experience in SEMs, such as in ecology, should consider and accept these limitations.

Improving Fit Indices in Structural Equation Modeling with Categorical Data

The Influence of Number of Categories and Threshold Values on Fit Indices in Structural Equation Modeling with Ordered Categorical Data

RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods

Structural Equation Modeling in the Context of Clinical Research

Fit Indices in Structural Equation Modeling and Confirmatory Factor Analysis: Reporting Guidelines

When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions.

Adapting fit indices for Bayesian structural equation modeling: Comparison to maximum likelihood.

A Primer of Model Fit Indices in Structural Equation Modeling.

Robust Estimation and Inference in Categorical Data

Evaluating Fit Indices for Multivariate t-Based Structural Equation Modeling with Data Contamination

The sensitivity of structural equation modeling with ordinal data to underlying non-normality and observed distributional forms.

On the Behavior of Fit Indices for Adjudicating Between Exploratory Structural Equation and Confirmatory Factor Analysis Models

Identifying latent classes with ordered categorical indicators

Dynamic fit index cutoffs for categorical factor analysis with Likert-type, ordinal, or binary responses.

Enhancing Model Fit Evaluation in SEM: Practical Tips for Optimizing Chi-Square Tests

Fit for a Bayesian: An Evaluation of PPP and DIC for Structural Equation Modeling

Equivalence Testing Based Fit Index: Standardized Root Mean Squared Residual

Assessing Fit in Ordinal Factor Analysis Models: SRMR vs. RMSEA

Impact of Informative Priors on Model Fit Indices in Bayesian Confirmatory Factor Analysis

How robust are Structural Equation Models to model miss-specification? A simulation study

Categorical exploratory data analysis on goodness-of-fit issues