Abstract:Scale validation is vital to psychological research because it ensures that scores from measurement scales represent the intended construct. Fit indices are commonly used to provide quantitative evidence that a proposed factor structure is plausible. However, there is a mismatch between guidelines for evaluating fit of the factor models and the data that most researchers have. Namely, fit guidelines are based on the simulations that assume item responses are collected on a continuous scale whereas most researchers collect discrete responses such as with a Likert-type scale. In this article, we show that common guidelines derived from assuming continuous responses (e.g., root-mean-square error of approximation 0.95) do not generalize to factor models applied to discrete responses. Specifically, discrete responses provide less information than continuous responses, so less information about misfit is passed to fit indices. Traditional guidelines, therefore, end up being too lenient and lose their ability to identify that a model may have a poor fit. We provide one possible solution by extending the recently developed dynamic fit index framework to accommodate discrete responses common in psychology. We conduct a simulation study to provide evidence that the proposed method consistently distinguishes between well-fitting and poorly fitting models. Results showed that our proposed cutoffs maintained at least 90% sensitivity to misspecification across studied conditions, whereas traditional cutoffs were highly inconsistent and frequently exhibited sensitivity below 50%. The proposed method is included in the dynamic R package and as a web-based Shiny application to make it easily accessible to psychologists. (PsycInfo Database Record (c) 2024 APA, all rights reserved)

dynamic : An R Package for Deriving Dynamic Fit Index Cutoffs for Factor Analysis

Dynamic fit index cutoffs for categorical factor analysis with Likert-type, ordinal, or binary responses.

Dynamic fit index cutoffs for one-factor models

Optimal Number of Replications for Obtaining Stable Dynamic Fit Index Cutoffs

Making model judgments ROC(K)-solid: Tailored cutoffs for fit indices through simulation and ROC analysis in structural equation modeling

The Poor Fit of Model Fit for Selecting Number of Factors in Exploratory Factor Analysis for Scale Evaluation

sparseDFM: An R Package to Estimate Dynamic Factor Models with Sparse Loadings

Parsimony in Model Selection: Tools for Assessing Fit Propensity

Equivalence Testing Based Fit Index: Standardized Root Mean Squared Residual

Decision Curve Analysis: a Technical Note

An R Function to Correct Fit Indices and Omnibus Tests in Confirmatory Factor Analysis

Model Fit and Item Factor Analysis: Overfactoring, Underfactoring, and a Program to Guide Interpretation

Fit Indices in Structural Equation Modeling and Confirmatory Factor Analysis: Reporting Guidelines

An evaluation of incremental fit indices: A clarification of mathematical and empirical properties

Adapting fit indices for Bayesian structural equation modeling: Comparison to maximum likelihood.

Cutoff criteria for overall model fit indexes in generalized structured component analysis

Evaluating Small Sample Approaches for Model Test Statistics in Structural Equation Modeling

Model-Selection-Based Approaches to Identifying the Optimal Number of Factors in Multilevel Exploratory Factor Analysis

Dynamic Sparse Factor Analysis

Make Some Noise: Generating Data from Imperfect Factor Models

Structural equation model testing: cutoff criteria for goodness of fit indices and chi-square test