Abstract:Several researchers have recommended that level-specific fit indices should be applied to detect the lack of model fit at any level in multilevel structural equation models. Although we concur with their view, we note that these studies did not sufficiently consider the impact of intraclass correlation (ICC) on the performance of level-specific fit indices. Our study proposed to fill this gap in the methodological literature. A Monte Carlo study was conducted to investigate the performance of (a) level-specific fit indices derived by a partially saturated model method (e.g., [Formula: see text] and [Formula: see text]) and (b) [Formula: see text] and [Formula: see text] in terms of their performance in multilevel structural equation models across varying ICCs. The design factors included intraclass correlation (ICC: ICC1 = 0.091 to ICC6 = 0.500), numbers of groups in between-level models (NG: 50, 100, 200, and 1,000), group size (GS: 30, 50, and 100), and type of misspecification (no misspecification, between-level misspecification, and within-level misspecification). Our simulation findings raise a concern regarding the performance of between-level-specific partial saturated fit indices in low ICC conditions: the performances of both [Formula: see text] and [Formula: see text] were more influenced by ICC compared with [Formula: see text] and SRMR B . However, when traditional cutoff values ( RMSEA≤ 0.06; CFI, TLI≥ 0.95; SRMR≤ 0.08) were applied, [Formula: see text] and [Formula: see text] were still able to detect misspecified between-level models even when ICC was as low as 0.091 (ICC1). On the other hand, both [Formula: see text] and [Formula: see text] were not recommended under low ICC conditions.

The Influence of Number of Categories and Threshold Values on Fit Indices in Structural Equation Modeling with Ordered Categorical Data

RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods

Improving Fit Indices in Structural Equation Modeling with Categorical Data

The Poor Fit of Model Fit for Selecting Number of Factors in Exploratory Factor Analysis for Scale Evaluation

Evaluating Fit Indices for Multivariate t-Based Structural Equation Modeling with Data Contamination

Effect of the Number of Variables on Measures of Fit in Structural Equation Modeling

Examining Chi-Square Test Statistics Under Conditions of Large Model Size and Ordinal Data

Confirmatory Factor Analysis of Ordinal Variables With Misspecified Models

Identifying latent classes with ordered categorical indicators

Model-Selection-Based Approaches to Identifying the Optimal Number of Factors in Multilevel Exploratory Factor Analysis

Assessing Fit in Ordinal Factor Analysis Models: SRMR vs. RMSEA

Fit Indices in Structural Equation Modeling and Confirmatory Factor Analysis: Reporting Guidelines

Structural Equation Modeling in the Context of Clinical Research

When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions.

Model Fit and Item Factor Analysis: Overfactoring, Underfactoring, and a Program to Guide Interpretation

The Impact of Intraclass Correlation on the Effectiveness of Level-Specific Fit Indices in Multilevel Structural Equation Modeling

Robust Estimation and Inference in Categorical Data

The sensitivity of structural equation modeling with ordinal data to underlying non-normality and observed distributional forms.

Dynamic fit index cutoffs for one-factor models

Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance

Categorizing a Continuous Predictor Subject to Measurement Error.