Abstract:Scale validation is vital to psychological research because it ensures that scores from measurement scales represent the intended construct. Fit indices are commonly used to provide quantitative evidence that a proposed factor structure is plausible. However, there is a mismatch between guidelines for evaluating fit of the factor models and the data that most researchers have. Namely, fit guidelines are based on the simulations that assume item responses are collected on a continuous scale whereas most researchers collect discrete responses such as with a Likert-type scale. In this article, we show that common guidelines derived from assuming continuous responses (e.g., root-mean-square error of approximation 0.95) do not generalize to factor models applied to discrete responses. Specifically, discrete responses provide less information than continuous responses, so less information about misfit is passed to fit indices. Traditional guidelines, therefore, end up being too lenient and lose their ability to identify that a model may have a poor fit. We provide one possible solution by extending the recently developed dynamic fit index framework to accommodate discrete responses common in psychology. We conduct a simulation study to provide evidence that the proposed method consistently distinguishes between well-fitting and poorly fitting models. Results showed that our proposed cutoffs maintained at least 90% sensitivity to misspecification across studied conditions, whereas traditional cutoffs were highly inconsistent and frequently exhibited sensitivity below 50%. The proposed method is included in the dynamic R package and as a web-based Shiny application to make it easily accessible to psychologists. (PsycInfo Database Record (c) 2024 APA, all rights reserved)

An R Function to Correct Fit Indices and Omnibus Tests in Confirmatory Factor Analysis

Fit Indices in Structural Equation Modeling and Confirmatory Factor Analysis: Reporting Guidelines

dynamic : An R Package for Deriving Dynamic Fit Index Cutoffs for Factor Analysis

A Primer of Model Fit Indices in Structural Equation Modeling.

Efficient Corrections for Standardized Person-Fit Statistics

The effect of latent and error non-normality on corrections to the test statistic in structural equation modeling

On the Behavior of Fit Indices for Adjudicating Between Exploratory Structural Equation and Confirmatory Factor Analysis Models

The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis.

Impact of Informative Priors on Model Fit Indices in Bayesian Confirmatory Factor Analysis

Model Fit and Item Factor Analysis: Overfactoring, Underfactoring, and a Program to Guide Interpretation

Enhancing Model Fit Evaluation in SEM: Practical Tips for Optimizing Chi-Square Tests

Equivalence Testing Based Fit Index: Standardized Root Mean Squared Residual

Dynamic fit index cutoffs for categorical factor analysis with Likert-type, ordinal, or binary responses.

Best practices for your confirmatory factor analysis: A JASP and lavaan tutorial

Adapting fit indices for Bayesian structural equation modeling: Comparison to maximum likelihood.

Evaluating Small Sample Approaches for Model Test Statistics in Structural Equation Modeling

Making model judgments ROC(K)-solid: Tailored cutoffs for fit indices through simulation and ROC analysis in structural equation modeling

The Poor Fit of Model Fit for Selecting Number of Factors in Exploratory Factor Analysis for Scale Evaluation

Dimensionality in confirmatory factor analysis is not in the eye of the beholder: Ancillary bifactor statistical indices illuminate dimensionality and reliability

Parsimony in Model Selection: Tools for Assessing Fit Propensity

Improvement of the GFI and AGFI indexes in structural equation models by the covariance between repetitions in the observed variables