Abstract:Scale validation is vital to psychological research because it ensures that scores from measurement scales represent the intended construct. Fit indices are commonly used to provide quantitative evidence that a proposed factor structure is plausible. However, there is a mismatch between guidelines for evaluating fit of the factor models and the data that most researchers have. Namely, fit guidelines are based on the simulations that assume item responses are collected on a continuous scale whereas most researchers collect discrete responses such as with a Likert-type scale. In this article, we show that common guidelines derived from assuming continuous responses (e.g., root-mean-square error of approximation 0.95) do not generalize to factor models applied to discrete responses. Specifically, discrete responses provide less information than continuous responses, so less information about misfit is passed to fit indices. Traditional guidelines, therefore, end up being too lenient and lose their ability to identify that a model may have a poor fit. We provide one possible solution by extending the recently developed dynamic fit index framework to accommodate discrete responses common in psychology. We conduct a simulation study to provide evidence that the proposed method consistently distinguishes between well-fitting and poorly fitting models. Results showed that our proposed cutoffs maintained at least 90% sensitivity to misspecification across studied conditions, whereas traditional cutoffs were highly inconsistent and frequently exhibited sensitivity below 50%. The proposed method is included in the dynamic R package and as a web-based Shiny application to make it easily accessible to psychologists. (PsycInfo Database Record (c) 2024 APA, all rights reserved)

Assessing Fit in Ordinal Factor Analysis Models: SRMR vs. RMSEA

Equivalence Testing Based Fit Index: Standardized Root Mean Squared Residual

The Poor Fit of Model Fit for Selecting Number of Factors in Exploratory Factor Analysis for Scale Evaluation

Confirmatory Factor Analysis of Ordinal Variables With Misspecified Models

RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods

Examining Chi-Square Test Statistics Under Conditions of Large Model Size and Ordinal Data

Fit Indices in Structural Equation Modeling and Confirmatory Factor Analysis: Reporting Guidelines

Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance

A Modification of McFadden's $R^2$ for Binary and Ordinal Response Models

The Influence of Number of Categories and Threshold Values on Fit Indices in Structural Equation Modeling with Ordered Categorical Data

Improving Fit Indices in Structural Equation Modeling with Categorical Data

On the Behavior of Fit Indices for Adjudicating Between Exploratory Structural Equation and Confirmatory Factor Analysis Models

Asymptotics of AIC, BIC, and RMSEA for Model Selection in Structural Equation Modeling

Dynamic fit index cutoffs for categorical factor analysis with Likert-type, ordinal, or binary responses.

Adapting fit indices for Bayesian structural equation modeling: Comparison to maximum likelihood.

Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares

Evaluating Small Sample Approaches for Model Test Statistics in Structural Equation Modeling

Alternative Ways of Assessing Model Fit

A Primer of Model Fit Indices in Structural Equation Modeling.

Dynamic fit index cutoffs for one-factor models

Model Fit and Item Factor Analysis: Overfactoring, Underfactoring, and a Program to Guide Interpretation