Konno aortoventriculoplasty in children and adolescents: from prosthetic valves to the Ross operation.

E. Erez,K. Kanter,V. Tam,W. Williams

DOI: https://doi.org/10.1016/S0003-4975(02)03595-6

2002-07-01

Abstract:

What problem does this paper attempt to address?

On a general theoretical framework of reliability

Yang Liu,Jolynn Pek,Alberto Maydeu‐Olivares

DOI: https://doi.org/10.1111/bmsp.12360

2024-10-17

British Journal of Mathematical and Statistical Psychology

Abstract:Reliability is an essential measure of how closely observed scores represent latent scores (reflecting constructs), assuming some latent variable measurement model. We present a general theoretical framework of reliability, placing emphasis on measuring the association between latent and observed scores. This framework was inspired by McDonald's (Psychometrika, 76, 511) regression framework, which highlighted the coefficient of determination as a measure of reliability. We extend McDonald's (Psychometrika, 76, 511) framework beyond coefficients of determination and introduce four desiderata for reliability measures (estimability, normalization, symmetry, and invariance). We also present theoretical examples to illustrate distinct measures of reliability and report on a numerical study that demonstrates the behaviour of different reliability measures. We conclude with a discussion on the use of reliability coefficients and outline future avenues of research.

psychology, experimental, mathematical,mathematics, interdisciplinary applications,statistics & probability
Robust Measurement Via A Fused Latent and Graphical Item Response Theory Model

Yunxiao Chen,Xiaoou Li,Jingchen Liu,Zhiliang Ying

DOI: https://doi.org/10.1007/s11336-018-9610-4

2018-01-01

Psychometrika

Abstract:Item response theory (IRT) plays an important role in psychological and educational measurement. Unlike the classical testing theory, IRT models aggregate the item level information, yielding more accurate measurements. Most IRT models assume local independence, an assumption not likely to be satisfied in practice, especially when the number of items is large. Results in the literature and simulation studies in this paper reveal that misspecifying the local independence assumption may result in inaccurate measurements and differential item functioning. To provide more robust measurements, we propose an integrated approach by adding a graphical component to a multidimensional IRT model that can offset the effect of unknown local dependence. The new model contains a confirmatory latent variable component, which measures the targeted latent traits, and a graphical component, which captures the local dependence. An efficient proximal algorithm is proposed for the parameter estimation and structure learning of the local dependence. This approach can substantially improve the measurement, given no prior information on the local dependence structure. The model can be applied to measure both a unidimensional latent trait and multidimensional latent traits.
Understanding Ability and Reliability Differences Measured with Count Items: The Distributional Regression Test Model and the Count Latent Regression Model

Marie Beisemann,Boris Forthmann,Philipp Doebler

DOI: https://doi.org/10.1080/00273171.2023.2288577

2024-02-14

Multivariate Behavioral Research

Abstract:In psychology and education, tests (e.g., reading tests) and self-reports (e.g., clinical questionnaires) generate counts, but corresponding Item Response Theory (IRT) methods are underdeveloped compared to binary data. Recent advances include the Two-Parameter Conway-Maxwell-Poisson model (2PCMPM), generalizing Rasch's Poisson Counts Model, with item-specific difficulty, discrimination, and dispersion parameters. Explaining differences in model parameters informs item construction and selection but has received little attention. We introduce two 2PCMPM-based explanatory count IRT models: The Distributional Regression Test Model for item covariates, and the Count Latent Regression Model for (categorical) person covariates. Estimation methods are provided and satisfactory statistical properties are observed in simulations. Two examples illustrate how the models help understand tests and underlying constructs.

psychology, experimental,mathematics, interdisciplinary applications,statistics & probability,social sciences, mathematical methods
Item Response Theory -- A Statistical Framework for Educational and Psychological Measurement

Yunxiao Chen,Xiaoou Li,Jingchen Liu,Zhiliang Ying

DOI: https://doi.org/10.48550/arXiv.2108.08604

2021-08-19

Abstract:Item response theory (IRT) has become one of the most popular statistical models for psychometrics, a field of study concerned with the theory and techniques of psychological measurement. The IRT models are latent factor models tailored to the analysis, interpretation, and prediction of individuals' behaviors in answering a set of measurement items that typically involve categorical response data. Many important questions of measurement are directly or indirectly answered through the use of IRT models, including scoring individuals' test performances, validating a test scale, linking two tests, among others. This paper provides a review of item response theory, including its statistical framework and psychometric applications. We establish connections between item response theory and related topics in statistics, including empirical Bayes, nonparametric methods, matrix completion, regularized estimation, and sequential analysis. Possible future directions of IRT are discussed from the perspective of statistical learning.

Methodology
A study of alternative approaches to non-normal latent trait distributions in item response theory models used for health outcome measurement

Niels Smits,Oğuzhan Öğreden,Mauricio Garnier-Villarreal,Caroline B Terwee,R Philip Chalmers

DOI: https://doi.org/10.1177/0962280220907625

IF: 2.494

2020-03-11

Statistical Methods in Medical Research

Abstract:It is often unrealistic to assume normally distributed latent traits in the measurement of health outcomes. If normality is violated, the item response theory (IRT) models that are used to calibrate questionnaires may yield parameter estimates that are biased. Recently, IRT models were developed for dealing with specific deviations from normality, such as zero-inflation (“excess zeros”) and skewness. However, these models have not yet been evaluated under conditions representative of item bank development for health outcomes, characterized by a large number of polytomous items. A simulation study was performed to compare the bias in parameter estimates of the graded response model (GRM), polytomous extensions of the zero-inflated mixture IRT (ZIM-GRM), and Davidian Curve IRT (DC-GRM). In the case of zero-inflation, the GRM showed high bias overestimating discrimination parameters and yielding estimates of threshold parameters that were too high and too close to one another, while ZIM-GRM showed no bias. In the case of skewness, the GRM and DC-GRM showed little bias with the GRM showing slightly better results. Consequences for the development of health outcome measures are discussed.

health care sciences & services,medical informatics,mathematical & computational biology,statistics & probability
Item Response Theory: A Modern Measurement Approach to Reliability and Precision for Counseling Researchers

Ryan M. Cook,Stefanie A. Wind

DOI: https://doi.org/10.1080/07481756.2023.2301284

2024-03-02

Measurement and Evaluation in Counseling and Development

Abstract:The purpose of this article is to discuss reliability and precision through the lens of a modern measurement approach, item response theory (IRT). Reliability evidence in the field of counseling is primarily generated using Classical Test Theory (CTT) approaches, although recent studies in the field of counseling have shown the benefits of using IRT approaches to explore measurement precision. We discuss the theoretical foundations and assumptions of CTT and IRT, and examine how modern measurement theory (i.e. IRT) poses advantages for capturing measurement precision. We use an example analysis to demonstrate the indices of measurement precision in IRT approaches (e.g. standard errors, person-fit, item-fit, targeting, residual-based fit statistics). Finally, we discuss practical and clinical implications for counseling researchers for using IRT approaches to measure precision, including insights into precision for persons, items, and measurement invariance as well as the utility of brief and adaptive scales. Using modern measurement theory approaches (i.e. item response theory), evidence of precision is generated for persons, items, and other relevant facets, and these indices can inform needed scale revisions, identify for whom the theorized model is a good fit, and the items that are well-targeted for the population of interest.

psychology, educational, applied
Understanding Reliability from a Regression Perspective

Yang Liu,Jolynn Pek,Alberto Maydeu-Olivares

DOI: https://doi.org/10.48550/arXiv.2404.16709

2024-10-28

Abstract:Reliability is an important quantification of measurement precision based on a latent variable measurement model. Inspired by McDonald (2011), we present a regression framework of reliability, placing emphasis on whether latent or observed scores serve as the regression outcome. Our theory unifies two extant perspectives of reliability: (a) classical test theory (measurement decomposition), and (b) optimal prediction of latent scores (prediction decomposition). Importantly, reliability should be treated as a property of the observed score under a measurement decomposition, but a property of the latent score under a prediction decomposition. To facilitate the evaluation and interpretation of distinct reliability coefficients for complex measurement models, we introduce a Monte Carlo approach for approximate calculation of reliability. We illustrate the proposed computational procedure with an empirical data analysis, which concerns measuring susceptibility and severity of depressive symptoms using a two-dimensional item response theory model. We conclude with a discussion on computing reliability coefficients and outline future avenues of research.

Methodology
Statistical foundations of person parameter estimation in the Thurstonian IRT model for forced‐choice and pairwise comparison designs

Safir Yousfi

DOI: https://doi.org/10.1111/bmsp.12364

2024-11-29

British Journal of Mathematical and Statistical Psychology

Abstract:The statistical foundations of person parameter estimation for the multivariate Thurstonian item response theory (TIRT) model of pairwise comparison and forced‐choice (FC) ranking data are elaborated, and several misconceptions in IRT and TIRT are addressed. It is shown that directional information (i.e. multivariate information as defined by Reckase & Kinley, 1991; Applied Psychological Measurement, 15, 361) is not suited to quantify the precision of the estimates unless the Fisher information matrix is diagonal. The asymptotic covariance can be quantified by the inverse Fisher information matrix if the genuine likelihood is used and by the inverse Godambe information for independence likelihood estimation that results from ignoring within‐block dependencies of pairwise comparisons. Analytical expressions are provided for the genuine likelihood and the Fisher information matrix for a generalized TIRT model that comprises binary pairwise comparison and ranking designs, which enables maximum likelihood estimation (MLE) and Bayesian estimation (maximum a posteriori probability with normal and Jeffreys prior) of person parameters. The bias of the MLE is quantified, and methods of bias prevention and bias correction are introduced. The correct marginal likelihood of graded pairwise comparisons is provided that might be used for person parameter estimation based on the independence likelihood.

psychology, experimental, mathematical,mathematics, interdisciplinary applications,statistics & probability
Alternative Approaches to Addressing Non-Normal Distributions in the Application of IRT Models to Personality Measures

Steven P. Reise,Anthony Rodriguez,Karen L. Spritzer,Ron D. Hays

DOI: https://doi.org/10.1080/00223891.2017.1381969

2017-10-31

Journal of Personality Assessment

Abstract:It is generally assumed that the latent trait is normally distributed in the population when estimating logistic item response theory (IRT) model parameters. This assumption requires that the latent trait be fully continuous and the population homogenous (i.e., not a mixture). When this normality assumption is violated, models are misspecified, and item and person parameter estimates are inaccurate. When normality cannot be assumed, it might be appropriate to consider alternative modeling approaches: (a) a zero-inflated mixture, (b) a log-logistic, (c) a Ramsay curve, or (d) a heteroskedastic-skew model. The first 2 models were developed to address modeling problems associated with so-called quasi-continuous or unipolar constructs, which apply only to a subset of the population, or are meaningful at one end of the continuum only. The second 2 models were developed to address non-normal latent trait distributions and violations of homogeneity of error variance, respectively. To introduce these alternative IRT models and illustrate their strengths and weaknesses, we performed real data application comparing results to those from a graded response model. We review both statistical and theoretical challenges in applying these models and choosing among them. Future applications of these and other alternative models (e.g., unfolding, diffusion) are needed to advance understanding about model choice in particular situations.

psychology, clinical, social
The Spearman-Brown Formula and Reliabilities of Random Test Forms

Jules L. Ellis

DOI: https://doi.org/10.48550/arXiv.2308.13811

2023-08-26

Methodology

Abstract:It is shown that the psychometric test reliability, based on any true-score model with randomly sampled items and conditionally independent errors, converges to 1 as the test length goes to infinity, assuming some fairly general regularity conditions. The asymptotic rate of convergence is given by the Spearman-Brown formula, and for this it is not needed that the items are parallel, or latent unidimensional, or even finite dimensional. Simulations with the 2-parameter logistic item response theory model reveal that there can be a positive bias in the reliability of short multidimensional tests, meaning that applying the Spearman-Brown formula in these cases would lead to overprediction of the reliability that will result from lengthening the tests. For short unidimensional tests under the 2-parameter logistic model the reliabilities are almost unbiased, meaning that application of the Spearman-Brown formula in these cases leads to predictions that are approximately unbiased.
An introduction to mixture item response theory models

R J De Ayala,S Y Santiago

DOI: https://doi.org/10.1016/j.jsp.2016.01.002

Abstract:Mixture item response theory (IRT) allows one to address situations that involve a mixture of latent subpopulations that are qualitatively different but within which a measurement model based on a continuous latent variable holds. In this modeling framework, one can characterize students by both their location on a continuous latent variable as well as by their latent class membership. For example, in a study of risky youth behavior this approach would make it possible to estimate an individual's propensity to engage in risky youth behavior (i.e., on a continuous scale) and to use these estimates to identify youth who might be at the greatest risk given their class membership. Mixture IRT can be used with binary response data (e.g., true/false, agree/disagree, endorsement/not endorsement, correct/incorrect, presence/absence of a behavior), Likert response scales, partial correct scoring, nominal scales, or rating scales. In the following, we present mixture IRT modeling and two examples of its use. Data needed to reproduce analyses in this article are available as supplemental online materials at http://dx.doi.org/10.1016/j.jsp.2016.01.002.
Redefining Item Response Models for Small Samples

Jean-Paul Fox

DOI: https://doi.org/10.3102/10769986241269886

2024-09-16

Journal of Educational and Behavioral Statistics

Abstract:Journal of Educational and Behavioral Statistics, Ahead of Print. Popular item response theory (IRT) models are considered complex, mainly due to the inclusion of a random factor variable (latent variable). The random factor variable represents the incidental parameter problem since the number of parameters increases when including data of new persons. Therefore, IRT models require a specific estimation method and large samples for accurate parameter estimation. The two-parameter IRT model is redefined by analytically integrating out the random person factor in the latent response formulation of the model to make it suitable for small sample applications. This IRT Bayesian covariance structure model (IRT-BCSM) describes the clustering of (latent) responses by persons through a structured covariance matrix in which a common covariance parameter represents the dependence implied by a unidimensional random factor variable. The IRT-BCSM is a latent-variable-free model and consists of (fixed) item parameters and a common covariance parameter, where person parameters can be post-hoc sampled. An efficient Gibbs sampler is proposed for parameter estimation. In simulation studies, the performance of the IRT-BCSM is compared to two-parameter IRT models for small samples, and results show an optimal performance of the IRT-BCSM even in sample sizes as small as 50 to 100 persons and 5 to 10 items. Generalizations of the IRT-BCSM show that a redefinition of more complex IRT models also lead to much more efficient parameterizations, which can broaden the scope of IRT applications.

education & educational research,psychology, mathematical,social sciences, mathematical methods
Modeling repeated self-reported outcome data: A continuous-time longitudinal Item Response Theory model

Cécile Proust-Lima,Viviane Philipps,Bastien Perrot,Myriam Blanchin,Véronique Sébille

DOI: https://doi.org/10.1016/j.ymeth.2022.01.005

IF: 4.647

2022-01-01

Methods

Abstract:Item Response Theory (IRT) models have received growing interest in health science for analyzing latent constructs such as depression, anxiety, quality of life or cognitive functioning from the information provided by each individual's items responses. However, in the presence of repeated item measures, IRT methods usually assume that the measurement occasions are made at the exact same time for all patients. In this paper, we show how the IRT methodology can be combined with the mixed model theory to provide a longitudinal IRT model which exploits the information of a measurement scale provided at the item level while simultaneously handling observation times that may vary across individuals and items. The latent construct is a latent process defined in continuous time that is linked to the observed item responses through a measurement model at each individual- and occasion-specific observation time; we focus here on a Graded Response Model for binary and ordinal items. The Maximum Likelihood Estimation procedure of the model is available in the R package lcmm. The proposed approach is contextualized in a clinical example in end-stage renal disease, the PREDIALA study. The objective is to study the trajectories of depressive symptomatology (as measured by 7 items of the Hospital Anxiety and Depression scale) according to the time from registration on the renal transplant waiting list and the renal replacement therapy. We also illustrate how the method can be used to assess Differential Item Functioning and lack of measurement invariance over time.

biochemistry & molecular biology,biochemical research methods
Differential Item Functioning via Robust Scaling

Peter F Halpin

DOI: https://doi.org/10.1007/s11336-024-09957-6

2024-05-04

Psychometrika

Abstract:This paper proposes a method for assessing differential item functioning (DIF) in item response theory (IRT) models. The method does not require pre-specification of anchor items, which is its main virtue. It is developed in two main steps: first by showing how DIF can be re-formulated as a problem of outlier detection in IRT-based scaling and then tackling the latter using methods from robust statistics. The proposal is a redescending M-estimator of IRT scaling parameters that is tuned to flag items with DIF at the desired asymptotic type I error rate. Theoretical results describe the efficiency of the estimator in the absence of DIF and its robustness in the presence of DIF. Simulation studies show that the proposed method compares favorably to currently available approaches for DIF detection, and a real data example illustrates its application in a research context where pre-specification of anchor items is infeasible. The focus of the paper is the two-parameter logistic model in two independent groups, with extensions to other settings considered in the conclusion.
Scalable Learning of Item Response Theory Models

Susanne Frick,Amer Krivošija,Alexander Munteanu

2024-08-15

Abstract:Item Response Theory (IRT) models aim to assess latent abilities of $n$ examinees along with latent difficulty characteristics of $m$ test items from categorical data that indicates the quality of their corresponding answers. Classical psychometric assessments are based on a relatively small number of examinees and items, say a class of $200$ students solving an exam comprising $10$ problems. More recent global large scale assessments such as PISA, or internet studies, may lead to significantly increased numbers of participants. Additionally, in the context of Machine Learning where algorithms take the role of examinees and data analysis problems take the role of items, both $n$ and $m$ may become very large, challenging the efficiency and scalability of computations. To learn the latent variables in IRT models from large data, we leverage the similarity of these models to logistic regression, which can be approximated accurately using small weighted subsets called coresets. We develop coresets for their use in alternating IRT training algorithms, facilitating scalable learning from large data.

Machine Learning,Data Structures and Algorithms
Comparing Attitudes Across Groups: An IRT-Based Item-Fit Statistic for the Analysis of Measurement Invariance

Janine Buchholz,Johannes Hartig

DOI: https://doi.org/10.1177/0146621617748323

Abstract:Questionnaires for the assessment of attitudes and other psychological traits are crucial in educational and psychological research, and item response theory (IRT) has become a viable tool for scaling such data. Many international large-scale assessments aim at comparing these constructs across countries, and the invariance of measures across countries is thus required. In its most recent cycle, the Programme for International Student Assessment (PISA 2015) implemented an innovative approach for testing the invariance of IRT-scaled constructs in the context questionnaires administered to students, parents, school principals, and teachers. On the basis of a concurrent calibration with equal item parameters across all groups (i.e., languages within countries), a group-specific item-fit statistic (root mean square deviance [RMSD]) was used as a measure for the invariance of item parameters for individual groups. The present simulation study examines the statistic's distribution under different types and extents of (non)invariance in polytomous items. Responses to five 4-point Likert-type items were generated under the generalized partial credit model (GPCM) for 1,000 simulees in 50 groups each. For one of the five items, either location or discrimination parameters were drawn from a normal distribution. In addition to the type of noninvariance, the extent of noninvariance was varied by manipulating the variation of these distributions. The results indicate that the RMSD statistic is better at detecting noninvariance related to between-group differences in item location than in item discrimination. The study's findings may be used as a starting point to sensitivity analysis aiming to define cutoff values for determining (non)invariance.
Introducing Flexible Monotone Multiple Choice Item Response Theory Models and Bit Scales

Joakim Wallmark,Maria Josefsson,Marie Wiberg

2024-10-02

Abstract:Item Response Theory (IRT) is a powerful statistical approach for evaluating test items and determining test taker abilities through response analysis. An IRT model that better fits the data leads to more accurate latent trait estimates. In this study, we present a new model for multiple choice data, the monotone multiple choice (MMC) model, which we fit using autoencoders. Using both simulated scenarios and real data from the Swedish Scholastic Aptitude Test, we demonstrate empirically that the MMC model outperforms the traditional nominal response IRT model in terms of fit. Furthermore, we illustrate how the latent trait scale from any fitted IRT model can be transformed into a ratio scale, aiding in score interpretation and making it easier to compare different types of IRT models. We refer to these new scales as bit scales. Bit scales are especially useful for models for which minimal or no assumptions are made for the latent trait scale distributions, such as for the autoencoder fitted models in this study.

Machine Learning,Methodology
Histological patterns of the metastases in pulmonary adenomatosis of sheep (jaagsiekte).

T. Nobel,F. Neumann,U. Klopfer

DOI: https://doi.org/10.1016/0021-9975(69)90074-7

IF: 1.083

1969-10-01

Journal of Comparative Pathology

Abstract:
Modeling Conditional Dependence of Response Accuracy and Response Time with the Diffusion Item Response Theory Model

Inhan Kang,Paul De Boeck,Roger Ratcliff

DOI: https://doi.org/10.1007/s11336-021-09819-5

2022-01-06

Psychometrika

Abstract:In this paper, we propose a model-based method to study conditional dependence between response accuracy and response time (RT) with the diffusion IRT model (Tuerlinckx and De Boeck in Psychometrika 70(4):629–650, 2005, https://doi.org/10.1007/s11336-000-0810-3; van der Maas et al. in Psychol Rev 118(2):339–356, 2011, https://doi.org/10.1080/20445911.2011.454498). We extend the earlier diffusion IRT model by introducing variability across persons and items in cognitive capacity (drift rate in the evidence accumulation process) and variability in the starting point of the decision processes. We show that the extended model can explain the behavioral patterns of conditional dependency found in the previous studies in psychometrics. Variability in cognitive capacity can predict positive and negative conditional dependency and their interaction with the item difficulty. Variability in starting point can account for the early changes in the response accuracy as a function of RT given the person and item effects. By the combination of the two variability components, the extended model can produce the curvilinear conditional accuracy functions that have been observed in psychometric data. We also provide a simulation study to validate the parameter recovery of the proposed model and present two empirical applications to show how to implement the model to study conditional dependency underlying data response accuracy and RTs.

psychology, mathematical,mathematics, interdisciplinary applications,social sciences, mathematical methods

Konno aortoventriculoplasty in children and adolescents: from prosthetic valves to the Ross operation.

On a general theoretical framework of reliability

Robust Measurement Via A Fused Latent and Graphical Item Response Theory Model

Understanding Ability and Reliability Differences Measured with Count Items: The Distributional Regression Test Model and the Count Latent Regression Model

Item Response Theory -- A Statistical Framework for Educational and Psychological Measurement

A study of alternative approaches to non-normal latent trait distributions in item response theory models used for health outcome measurement

Item Response Theory: A Modern Measurement Approach to Reliability and Precision for Counseling Researchers

Understanding Reliability from a Regression Perspective

Statistical foundations of person parameter estimation in the Thurstonian IRT model for forced‐choice and pairwise comparison designs

Alternative Approaches to Addressing Non-Normal Distributions in the Application of IRT Models to Personality Measures

The Spearman-Brown Formula and Reliabilities of Random Test Forms

An introduction to mixture item response theory models

Redefining Item Response Models for Small Samples

Modeling repeated self-reported outcome data: A continuous-time longitudinal Item Response Theory model

Differential Item Functioning via Robust Scaling

Scalable Learning of Item Response Theory Models

Comparing Attitudes Across Groups: An IRT-Based Item-Fit Statistic for the Analysis of Measurement Invariance

Introducing Flexible Monotone Multiple Choice Item Response Theory Models and Bit Scales

Histological patterns of the metastases in pulmonary adenomatosis of sheep (jaagsiekte).

Modeling Conditional Dependence of Response Accuracy and Response Time with the Diffusion Item Response Theory Model