Abstract:Exploratory factor analyses are commonly used to determine the underlying factors of multiple observed variables. Many criteria have been suggested to determine how many factors should be retained. In this study, we present an extensive Monte Carlo simulation to investigate the performance of extraction criteria under varying sample sizes, numbers of indicators per factor, loading magnitudes, underlying multivariate distributions of observed variables, as well as how the performance of the extraction criteria are influenced by the presence of cross-loadings and minor factors for unidimensional, orthogonal, and correlated factor models. We compared several variants of traditional parallel analysis (PA), the Kaiser-Guttman Criterion, and sequential χ2 model tests (SMT) with 4 recently suggested methods: revised PA, comparison data (CD), the Hull method, and the Empirical Kaiser Criterion (EKC). No single extraction criterion performed best for every factor model. In unidimensional and orthogonal models, traditional PA, EKC, and Hull consistently displayed high hit rates even in small samples. Models with correlated factors were more challenging, where CD and SMT outperformed other methods, especially for shorter scales. Whereas the presence of cross-loadings generally increased accuracy, non-normality had virtually no effect on most criteria. We suggest researchers use a combination of SMT and either Hull, the EKC, or traditional PA, because the number of factors was almost always correctly retrieved if those methods converged. When the results of this combination rule are inconclusive, traditional PA, CD, and the EKC performed comparatively well. However, disagreement also suggests that factors will be harder to detect, increasing sample size requirements to N ≥ 500. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Double Cross Validation for the Number of Factors in Approximate Factor Models

Determining the Number of Common Functional Factors with Twice Cross-Validation

Cross-validatory Framework for Optimal Parameter Estimation of KPCA and KPLS Models

Bi-cross-validation for factor analysis

Iterative Approximate Cross-Validation

A Testing Procedure for Determining the Number of Factors in Approximate Factor Models With Large Datasets

Approximate Cross-validation: Guarantees for Model Assessment and Selection

A Multi-Step Procedure to Determine the Number of Factors in Large Approximate Factor Models

On Factor Models with Random Missing: EM Estimation, Inference, and Cross Validation

Simultaneous Inference For High-Dimensional Approximate Factor Model

Additional Online Supplement for “Inference for Approximate Factor Models: Random Missing and Cross Validation”

How to determine the number of factors to retain in exploratory factor analysis: A comparison of extraction methods under realistic conditions.

Model Selection Via Multifold Cross Validation

Efficient leave-one-out cross-validation for Bayesian non-factorized normal and Student-t models

Bi-cross-validation of the SVD and the nonnegative matrix factorization

Is Cross-Validation the Gold Standard to Evaluate Model Performance?

Cross-validation: what does it estimate and how well does it do it?

One model to rule them all? Using machine learning algorithms to determine the number of factors in exploratory factor analysis.

Bootstrapping the Cross-Validation Estimate

Cross-validation of component models: A critical look at current methods

Determining the number of factors in a large-dimensional generalised factor model