Abstract:Many studies in fields such as psychology and educational sciences obtain information about attributes of subjects through observational studies, in which raters score subjects using multiple-item rating scales. Error variance due to measurement effects, such as items and raters, attenuate the regression coefficients and lower the power of (hierarchical) linear models. A modeling procedure is discussed to reduce the attenuation. The procedure consists of (1) an item response theory (IRT) model to map the discrete item responses to a continuous latent scale and (2) a generalizability theory (GT) model to separate the variance in the latent measurement into variance components of interest and nuisance variance components. It will be shown how measurements obtained from this mixture of IRT and GT models can be embedded in (hierarchical) linear models, both as predictor or criterion variables, such that error variance due to nuisance effects are partialled out. Using examples from the field of educational measurement, it is shown how general-purpose software can be used to implement the modeling procedure.

What problem does this paper attempt to address?

### The Problem the Paper Attempts to Solve The paper aims to address the issue of attenuation caused by measurement errors in regression analysis of rating data obtained through observational studies in fields such as psychology and educational sciences. Specifically, researchers often use multi-item rating scales to assess certain attributes of subjects, and these scales typically include scores given by different raters. In such cases, measurement errors can reduce the validity and statistical power of regression coefficients. The paper proposes a method that combines Item Response Theory (IRT) and Generalizability Theory (GT) to mitigate the impact of measurement errors. The specific steps are as follows: 1. **Item Response Theory (IRT) Model**: Maps discrete item responses to a continuous latent scale. 2. **Generalizability Theory (GT) Model**: Separates the variance components in the latent measurement, distinguishing between the variance components of interest and the variance components of nuisance factors. This method allows the embedding of measurements obtained from the combination of IRT and GT models into (hierarchical) linear models, thereby excluding error variance caused by nuisance effects. An example from the field of educational measurement demonstrates how to implement this modeling process using general-purpose software. ### Summary The core objective of the paper is to improve the results of regression analysis based on rating scale data by combining IRT and GT models, reducing the impact of measurement errors on regression coefficients, and enhancing the reliability and validity of the research.

Reducing Attenuation Bias in Regression Analyses Involving Rating Scale Data via Psychometric Modeling

Inflated Discrete Beta Regression Models for Likert and Discrete Rating Scale Outcomes

The Stabilizing Influences of Linking Set Size and Model-Data Fit in Sparse Rater-Mediated Assessment Networks

Modeling Item-Level Heterogeneous Treatment Effects With the Explanatory Item Response Model: Leveraging Large-Scale Online Assessments to Pinpoint the Impact of Educational Interventions

Introducing Flexible Monotone Multiple Choice Item Response Theory Models and Bit Scales

Psychometric approaches for developing commensurate measures across independent studies: Traditional and new models.

A Causal View on Bias in Missing Data Imputation: The Impact of Evil Auxiliary Variables on Norming of Test Scores

A Latent Trait Model for Multivariate Longitudinal Data With Two Sources of Measurement Error

Benefits of the Curious Behavior of Bayesian Hierarchical Item Response Theory Models—An in-Depth Investigation and Bias Correction

A Note on Evaluation of Polytomous Item Locations With the Rating Scale Model and Testing Its Fit

Modeling Response Styles in Cross‐Classified Data Using a Cross‐Classified Multidimensional Nominal Response Model

Alternative Approaches to Addressing Non-Normal Distributions in the Application of IRT Models to Personality Measures

Using Generalizability Theory to Disattenuate Correlation Coefficients for Multiple Sources of Measurement Error

Robust Estimation and Inference in Categorical Data

Exploring Within-Rater Category Ordering: A Simulation Study Using Adjacent-Categories Mokken Scale Analysis

Effects of the Quantity and Magnitude of Cross-Loading and Model Specification on MIRT Item Parameter Recovery

Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations

A Psychometric Perspective on the Associations between Response Accuracy and Response Time Residuals

A study of alternative approaches to non-normal latent trait distributions in item response theory models used for health outcome measurement

A psychometric modeling approach to fuzzy rating data