Abstract:Background: Increasing attention is being given to assessing treatment effect heterogeneity among individuals belonging to qualitatively different latent subgroups. Inference routinely proceeds by first partitioning the individuals into subgroups, then estimating the subgroup-specific average treatment effects. However, because the subgroups are only latently associated with the observed variables, the actual individual subgroup memberships are rarely known with certainty in practice and thus have to be imputed. Ignoring the uncertainty in the imputed memberships precludes misclassification errors, potentially leading to biased results and incorrect conclusions. Methods: We propose a strategy for assessing the sensitivity of inference to classification uncertainty when using such classify-analyze approaches for subgroup effect analyses. We exploit each individual's typically nonzero predictive or posterior subgroup membership probabilities to gauge the stability of the resultant subgroup-specific average causal effects estimates over different, carefully selected subsets of the individuals. Because the membership probabilities are subject to sampling variability, we propose Monte Carlo confidence intervals that explicitly acknowledge the imprecision in the estimated subgroup memberships via perturbations using a parametric bootstrap. The proposal is widely applicable and avoids stringent causal or structural assumptions that existing bias-adjustment or bias-correction methods rely on. Results: Using two different publicly available real-world datasets, we illustrate how the proposed strategy supplements existing latent subgroup effect analyses to shed light on the potential impact of classification uncertainty on inference. First, individuals are partitioned into latent subgroups based on their medical and health history. Then within each fixed latent subgroup, the average treatment effect is assessed using an augmented inverse propensity score weighted estimator. Finally, utilizing the proposed sensitivity analysis reveals different subgroup-specific effects that are mostly insensitive to potential misclassification. Conclusions: Our proposed sensitivity analysis is straightforward to implement, provides both graphical and numerical summaries, and readily permits assessing the sensitivity of any machine learning-based causal effect estimator to classification uncertainty. We recommend making such sensitivity analyses more routine in latent subgroup effect analyses.

Nonequivalence of measurement in latent variable modeling of multigroup data: A sensitivity analysis.

Testing the Measurement Equivalence of Personality Adjective Items Across Cultures

Testing measurement invariance across groups: applications in cross-cultural research.

Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance

The Necessity of Testing Measurement Invariance in Cross-Cultural Research: Potential Bias in Cross-Cultural Comparisons With Individualism– Collectivism Self-Report Scales

Measuring up: the significance of measurement invariance in cardiovascular research

A Comparison of FIML- versus Multiple-imputation-based methods to test measurement invariance with incomplete ordinal variables

Testing Measurement Invariance Across Unobserved Groups: The Role of Covariates in Factor Mixture Modeling

Measurement Invariance and Differential Item Functioning in Latent Class Analysis With Stepwise Multiple Indicator Multiple Cause Modeling

Evaluating sensitivity to classification uncertainty in latent subgroup effect analyses

Single-Item Versus Multiple-Item Measurement Scales: An Empirical Comparison

Psychometric approaches for developing commensurate measures across independent studies: Traditional and new models.

Measurement equivalence: A non-technical primer on categorical multi-group confirmatory factor analysis in school psychology

The dire disregard of measurement invariance testing in psychological science.

Effects of language of assessment on the measurement of acculturation: Measurement equivalence and cultural frame switching.

Equivalence testing for multiple groups

A Comparison of Confirmatory Factor Analysis and Network Models for Measurement Invariance Assessment When Indicator Residuals are Correlated

Necessary but Insufficient: Why Measurement Invariance Tests Need Online Probing as a Complementary Tool

On Detecting Systematic Measurement Error in Cross-Cultural Research: A Review and Critical Reflection on Equivalence and Invariance Tests

The measurement equivalence of instruments and its application in nursing research

Evaluating Measurement Invariance in Categorical Data Latent Variable Models with the EPC-Interest