Abstract:Partial least squares path modeling (PLS-PM) has become popular in various disciplines to model structural relationships among latent variables measured by manifest variables. To fully benefit from the predictive capabilities of PLS-PM, researchers must understand the efficacy of predictive metrics used. In this research, we compare the performance of standard PLS-PM criteria and model selection criteria derived from Information Theory, in terms of selecting the best predictive model among a cohort of competing models. We use Monte Carlo simulation to study this question under various sample sizes, effect sizes, item loadings, and model setups. Specifically, we explore whether, and when, the in-sample measures such as the model selection criteria can substitute for out-of-sample criteria that require a holdout sample. Such a substitution is advantageous when creating a holdout causes considerable loss of statistical and predictive power due to an overall small sample. We find that when the researcher does not have the luxury of a holdout sample, and the goal is selecting correctly specified models with low prediction error, the in-sample model selection criteria, in particular the Bayesian Information Criterion (BIC) and Geweke-Meese Criterion (GM), are useful substitutes for out-of-sample criteria. When a holdout sample is available, the best performing out-of-sample criteria include the root mean squared error (RMSE) and mean absolute deviation (MAD). We recommend against using standard the PLS-PM criteria (R-2, Adjusted R-2, and Q(2)), and specifically the out-of-sample mean absolute percentage error (MAPE) for prediction-oriented model selection purposes. Finally, we illustrate the model selection criteria's practical utility using a well-known corporate reputation model.

Can We Differentiate a Latent Growth Curve Model from Competitors? Evidence Based on Individual Case Residuals

Detecting Growth Shape Misspecifications in Latent Growth Models: An Evaluation of Fit Indexes

Evaluating fit indices in a multilevel latent growth model with unbalanced design: a Monte Carlo study

Latent Growth Curve Models

Interaction Effects in Latent Growth Models: Evaluation of Alternative Estimation Approaches

Bayesian Methods and Model Selection for Latent Growth Curve Models with Missing Data

Fit Indices for Mean Structures with Growth Curve Models.

Model Selection Criteria for Latent Growth Models Using Bayesian Methods

Bayesian Approach to Non-ignorable Missingness in Latent Growth Models

Empirical-likelihood-based criteria for model selection on marginal analysis of longitudinal data with dropout missingness

The Effect of Model Misspecification on Growth Mixture Model Class Enumeration

A Cautionary Note on Identification and Scaling Issues in Second-order Latent Growth Models

On the Correspondence Between the Latent Growth Curve and Latent Change Score Models

Selecting a linear mixed model for longitudinal data: Repeated measures analysis of variance, covariance pattern model, and growth curve approaches.

Item-Weighted Likelihood Method for Measuring Growth in Longitudinal Study With Tests Composed of Both Dichotomous and Polytomous Items

Jenss-Bayley Latent Change Score Model with Individual Ratio of Growth Acceleration in the Framework of Individual Measurement Occasions

Evaluating the Relative Merits of Competing Models Based on Empirical Likelihood Ratio Test

Methods Comparison of Piecewise Growth Modeling

Promote Sign Consistency in Cure Rate Model with Weibull Lifetime

Prediction-Oriented Model Selection In Partial Least Squares Path Modeling

Latent Growth Curve and Repeated Measures ANOVA Contrasts: What the Models are Telling You