The interactive effects of nocturnal sleep and daytime naps in relation to serum C-reactive protein.

J. Mantua,R. Spencer

DOI: https://doi.org/10.1016/j.sleep.2015.06.014

IF: 4.842

2015-10-01

Sleep Medicine

Abstract:

What problem does this paper attempt to address?

Developing an accurate and efficient tool for the internalizing spectrum: A simulation study of the adaptive algorithm to the Inventory of Depression and Anxiety Symptoms II (IDAS-II)

M Sanchez-Garcia,A De la Rosa-Cáceres,G Rossi,C Diaz-Batanero

DOI: https://doi.org/10.1002/mpr.2032

Abstract:Objectives: This research simulates an adaptive version of the IDAS-II (IDAS-CAT). Methods: 2021 participants from both community (n = 1692) and patients (n = 329) samples completed the IDAS-II. Item response theory metric properties of the IDAS-II full test and the 20-items of the general depression (GD) scale were obtained. The efficiency and accuracy of different computerized adaptive algorithms were simulated. Different subsamples completed additional external measures in order to gather evidence of validity of the scores estimated with the simulated adaptive algorithms selected. Results: Both unidimensional computerized adaptive testing algorithm selected for the GD scale and the bifactor model chosen for the full test, allow 70% reduction in the length of administration, maintaining a measurement error below 0.30 on the general and 0.50 on the specific factors. Results show high correlations of the scores estimated with the adaptive algorithms and the estimates based on the full test, as well as correlations with external criteria almost equal to those generated with the full test. Conclusions: IDAS-CAT could be a reliable and fast tool for measuring internalizing spectrum.
Validation of the structured interview section of the penn computerized adaptive test for neurocognitive and clinical psychopathology assessment (CAT GOASSESS)

Eirini Zoupou,Tyler M Moore,Kelly P Kennedy,Monica E Calkins,Alesandra Gorgone,Akira Di Sandro,Sage Rush,Katherine C Lopez,Kosha Ruparel,Tarlan Daryoush,Paul Okoyeh,Andrew Savino,Scott Troyan,Daniel H Wolf,J Cobb Scott,Raquel E Gur,Ruben C Gur,Tyler M. Moore,Kelly P. Kennedy,Monica E. Calkins,Katherine C. Lopez,Daniel H. Wolf,J. Cobb Scott,Raquel E. Gur,Ruben C. Gur

DOI: https://doi.org/10.1016/j.psychres.2024.115862

IF: 11.3

2024-05-01

Psychiatry Research

Abstract:Large-scale studies and burdened clinical settings require precise, efficient measures that assess multiple domains of psychopathology. Computerized adaptive tests (CATs) can reduce administration time without compromising data quality. We examined feasibility and validity of an adaptive psychopathology measure, GOASSESS, in a clinical community-based sample (N = 315; ages 18-35) comprising three groups: healthy controls, psychosis, mood/anxiety disorders. Assessment duration was compared between the Full and CAT GOASSESS. External validity was tested by comparing how the CAT and Full versions related to demographic variables, study group, and socioeconomic status. The relationships between scale scores and criteria were statistically compared within a mixed-model framework to account for dependency between relationships. Convergent validity was assessed by comparing scores of the CAT and the Full GOASSESS using Pearson correlations. The CAT GOASSESS reduced interview duration by more than 90 % across study groups and preserved relationships to external criteria and demographic variables as the Full GOASSESS. All CAT GOASSESS scales could replace those of the Full instrument. Overall, the CAT GOASSESS showed acceptable psychometric properties and demonstrated feasibility by markedly reducing assessment time compared to the Full GOASSESS. The adaptive version could be used in large-scale studies or clinical settings for intake screening.

psychiatry
Development and public release of a computerized adaptive (CAT) version of the Schizotypal Personality Questionnaire

Tyler M Moore,Monica E Calkins,Steven P Reise,Ruben C Gur,Raquel E Gur

DOI: https://doi.org/10.1016/j.psychres.2018.02.022

Abstract:One of the most widely used measures of psychosis-related symptoms and characteristics is the 74-item Schizotypal Personality Questionnaire (SPQ). Using multidimensional Item Response Theory (bifactor 2-parameter model), we calibrated SPQ items in a sample of 375 youths aged 9-24 years and constructed a fully functional computerized adaptive form of the SPQ on an open-source platform for public use. To assess validity, we used the above parameters to simulate CAT sessions in a separate validation sample (N = 100) using three test-length-based stopping rules: 8 items, 16 items, and 32 items. Those scores were then compared to full-form and SPQ-Brief scores on their abilities to predict psychosis or clinical risk status. Areas under the receiver operating characteristic curves indicated mediocre predictive ability, but did not differ among any of the forms, even when only eight adaptive items were administered. The Youden index for the 16-item adaptive version was higher than that for the 22-item SPQ-Brief. Classification accuracy for the full SPQ was 73% compared to 66% for the both the SPQ-Brief and adaptive versions (average of three stopping rules). The SPQ-CAT shows promise as a much shorter but valid assessment of schizotypy which can save time with minimal loss of information.
Computerized Adaptive Testing for Sleep Disorders: Development of An Item Bank and Validation in A Simulated Study

Menghua She,Yaling Li,Dongbo Tu,Yan Cai

DOI: https://doi.org/10.21203/rs.3.rs-18576/v1

2020-04-22

Abstract:Abstract Background : As more and more people suffer from sleep disorders, developing an efficient, cheap and accurate assessment tool for screening sleep disorders is becoming more urgent. This study developed a computerized adaptive testing for sleep disorders (CAT-SD). Methods : A large sample of 1,304 participants was recruited to construct the item pool of CAT-SD and to investigate the psychometric characteristics of CAT-SD. More specifically, firstly the analyses of unidimensionality, model fit, item fit, item discrimination parameter and differential item functioning (DIF) were conducted to construct a final item pool which meets the requirements of item response theory (IRT) measurement. In addition, a simulated CAT study with real response data of participants was performed to investigate the psychometric characteristics of CAT-SD, including reliability, validity and predictive utility (sensitivity and specificity). Results: The final unidimensional item bank of the CAT-SD not only had good item fit, high discrimination and no DIF; Moreover, it had acceptable reliability, validity and predictive utility. Conclusions : The CAT-SD could be used as an effective and accurate assessment tool for measuring individuals' severity of the sleep disorders and offers a bran-new perspective for screening of sleep disorders with psychological scales.
Computerized adaptive testing to screen pre-school children for emotional and behavioral problems

Meinou H. C. Theunissen,Iris Eekhout,Sijmen A. Reijneveld

DOI: https://doi.org/10.1007/s00431-023-05414-1

2024-01-23

European Journal of Pediatrics

Abstract:Questionnaires to detect emotional and behavioral (EB) problems in preventive child healthcare (PCH) should be short; this potentially affects their validity and reliability. Computerized adaptive testing (CAT) could overcome this weakness. The aim of this study was to (1) develop a CAT to measure EB problems among pre-school children and (2) assess the efficiency and validity of this CAT. We used a Dutch national dataset obtained from parents of pre-school children undergoing a well-child care assessment by PCH ( n = 2192, response 70%). Data regarded 197 items on EB problems, based on four questionnaires, the Strengths and Difficulties Questionnaire (SDQ), the Child Behavior Checklist (CBCL), the Ages and Stages Questionnaire: Social Emotional (ASQ:SE), and the Brief Infant–Toddler Social and Emotional Assessment (BITSEA). Using 80% of the sample, we calculated item parameters necessary for a CAT and defined a cutoff for EB problems. With the remaining part of the sample, we used simulation techniques to determine the validity and efficiency of this CAT, using as criterion a total clinical score on the CBCL. Item criteria were met by 193 items. This CAT needed, on average, 16 items to identify children with EB problems. Sensitivity and specificity compared to a clinical score on the CBCL were 0.89 and 0.91, respectively, for total problems; 0.80 and 0.93 for emotional problems; and 0.94 and 0.91 for behavioral problems.

pediatrics
Initial Validation of a Computerized Adaptive Test for Substance Use Disorder Identification in Adolescents

Zachary W. Adams,Leslie A. Hulvershorn,Michael P. Smoker,Brigid R. Marriott,Matthew C. Aalsma,Robert D. Gibbons

DOI: https://doi.org/10.1080/10826084.2024.2305801

2024-01-27

Substance Use & Misuse

Abstract:Purpose Computerized adaptive tests (CATs) are highly efficient assessment tools that couple low patient and clinician time burden with high diagnostic accuracy. A CAT for substance use disorders (CAT-SUD-E) has been validated in adult populations but has yet to be tested in adolescents. The purpose of this study was to perform initial evaluation of the K-CAT-SUD-E (i.e., Kiddy-CAT-SUD-E) in an adolescent sample compared to a gold-standard diagnostic interview.

psychiatry,substance abuse,psychology
Developing Computerized Adaptive Testing for a National Health Professionals Exam: An Attempt from Psychometric Simulations

Lingling Xu,Zhehan Jiang,Yuting Han,Haiying Liang,Jinying Ouyang

DOI: https://doi.org/10.5334/pme.855

2023-10-31

Abstract:Introduction: The accurate assessment of health professionals' competence is critical for ensuring public health safety and quality of care. Computerized Adaptive Testing (CAT) based on the Item Response Theory (IRT) has the potential to improve measurement accuracy and reduce respondent burden. In this study, we conducted psychometric simulations to develop a CAT for evaluating the candidates' competence of health professionals. Methods: The initial CAT item bank was sourced from the Standardized Competence Test for Clinical Medicine Undergraduates (SCTCMU), a nationwide summative test in China, consisting of 300 multiple-choice items. We randomly selected response data from 2000 Chinese clinical medicine undergraduates for analysis. Two types of analyses were performed: first, evaluating the psychometric properties of all items to meet the requirements of CAT; and second, conducting multiple CAT simulations using both simulated and real response data. Results: The final CAT item bank consisted of 121 items, for which item parameters were calculated using a two-parameter logistic model (2PLM). The CAT simulations, based on both simulated and real data, revealed sufficient marginal reliability (coefficient of marginal reliability above 0.750) and criterion-related validity (Pearson's correlations between CAT scores and aggregate scores of the SCTCMU exceeding 0.850). Discussion: In national-level medical education assessment, there is an increasing need for concise yet valid evaluations of candidates' competence of health professionals. The CAT developed in this study demonstrated satisfactory reliability and validity, offering a more efficient assessment of candidates' competence of health professionals. The psychometric properties of the CAT could lead to shorter test durations, reduced information loss, and a decreased testing burden for participants.
Efficiency of computerized adaptive testing with a cognitively designed item bank

Hao Luo,Xiangdong Yang

DOI: https://doi.org/10.3389/fpsyg.2024.1353419

IF: 3.8

2024-06-26

Frontiers in Psychology

Abstract:An item bank is key to applying computerized adaptive testing (CAT). The traditional approach to developing an item bank requires content experts to design each item individually, which is a time-consuming and costly process. The cognitive design system (CDS) approach offers a solution by automating item generation. However, the CDS approach has a specific way of calibrating or predicting item difficulty that affects the measurement efficiency of CAT. A simulation study was conducted to compare the efficiency of CAT using both calibration and prediction models. The results show that, although the predictive model (linear logistic trait model; LLTM) shows a higher root mean square error (RMSE) than the baseline model (Rasch), it requires only a few additional items to achieve comparable RMSE. Importantly, the number of additional items needed decreases as the explanatory rate of the model increases. These results indicate that the slight reduction in measurement efficiency due to prediction item difficulty is acceptable. Moreover, the use of prediction item difficulty can significantly reduce or even eliminate the need for item pretesting, thereby reducing the costs associated with item calibration.

psychology, multidisciplinary
Development of a Computerized Adaptive Testing for ADHD Using Bayesian Networks: an Attempt at Classification

Zhehan Jiang,Wenchao Ma,Kate Flory,Dan Zhang,Wenjing Zhou,Dexin Shi,Xiaochen Hua,Ren Liu

DOI: https://doi.org/10.1007/s12144-022-03056-4

IF: 2.8

2022-01-01

Current Psychology

Abstract:Quantifying the severity of neurological disorders, such as attention-deficit/hyperactivity disorder (ADHD), is encouraged in practice. To explore the methodological properties of data-driven classification, this study aimed to develop computerized adaptive testing (CAT) based on Bayesian networks to classify ADHD symptom severity for individual subjects efficiently. The Current Symptoms Scale was used to build an ADHD symptom severity classifier in a sample of 892 college students, and the CAT making use of information gains from conditional probabilities of the Bayesian network was evaluated through a simulation study. The results showed that distributing 8 out of 18 items on average to an individual subject would result in 85% classification accuracy, supporting the adaption of surveying ADHD to digital platforms demanding much-shortened scales. The severity classifier based on Bayesian networks can assist diagnostic tasks via a data-driven mechanism. Further, integrating CAT into the classifier can shorten the scale while maintain acceptable measurement quality. The ADHD symptoms were used in the paper; however, the proposed method applies to other psychological symptoms as well. The impact and limitations of the CAT for ADHD were discussed, and several suggestions for future research were provided.
The Comprehensive Autistic Trait Inventory (CATI): development and validation of a new measure of autistic traits in the general population

Michael C. W. English,Gilles E. Gignac,Troy A. W. Visser,Andrew J. O. Whitehouse,James T. Enns,Murray T. Maybery

DOI: https://doi.org/10.1186/s13229-021-00445-7

2021-05-17

Molecular Autism

Abstract:Abstract Background Traits and characteristics qualitatively similar to those seen in diagnosed autism spectrum disorder can be found to varying degrees in the general population. To measure these traits and facilitate their use in autism research, several questionnaires have been developed that provide broad measures of autistic traits [e.g. Autism-Spectrum Quotient (AQ), Broad Autism Phenotype Questionnaire (BAPQ)]. However, since their development, our understanding of autism has grown considerably, and it is arguable that existing measures do not provide an ideal representation of the trait dimensions currently associated with autism. Our aim was to create a new measure of autistic traits that reflects our current understanding of autism, the Comprehensive Autism Trait Inventory (CATI). Methods In Study 1, 107 pilot items were administered to 1166 individuals in the general population and exploratory factor analysis of responses used to create the 42-item CATI comprising six subscales: Social Interactions , Communication , Social Camouflage , Repetitive Behaviours , Cognitive Rigidity , and Sensory Sensitivity . In Study 2, the CATI was administered to 1119 new individuals and confirmatory factor analysis used to verify the factor structure. The AQ and BAPQ were administered to validate the CATI, and additional autistic participants were recruited to compare the predictive ability of the measures. In Study 3, to validate the CATI subscales, the CATI was administered to 202 new individuals along with existing valid measures qualitatively similar to each CATI subscale. Results The CATI showed convergent validity at both the total-scale ( r ≥ .79) and subscale level ( r ≥ .68). The CATI also showed superior internal reliability for total-scale scores ( α = .95) relative to the AQ ( α = .90) and BAPQ ( α = .94), consistently high reliability for subscales ( α > .81), greater predictive ability for classifying autism (Youden’s Index = .62 vs .56–.59), and demonstrated measurement invariance for sex. Limitations Analyses of predictive ability for classifying autism depended upon self-reported diagnosis or identification of autism. The autistic sample was not large enough to test measurement invariance of autism diagnosis. Conclusions The CATI is a reliable and economical new measure that provides observations across a wide range of trait dimensions associated with autism, potentially precluding the need to administer multiple measures, and to our knowledge, the CATI is also the first broad measure of autistic traits to have dedicated subscales for social camouflage and sensory sensitivity.

genetics & heredity,neurosciences
Psychometric Assessment of an Item Bank for Adaptive Testing on Patient-Reported Experience of Care Environment for Severe Mental Illness: Validation Study

Sara Fernandes,Yann Brousse,Xavier Zendjidjian,Delphine Cano,Jérémie Riedberger,Pierre-Michel Llorca,Ludovic Samalin,Daniel Dassa,Christian Trichard,Vincent Laprevote,Anne Sauvaget,Mocrane Abbar,David Misdrahi,Fabrice Berna,Christophe Lancon,Nathalie Coulon,Wissam El-Hage,Pierre-Emmanuel Rozier,Michel Benoit,Bruno Giordana,Alejandra Caqueo-Urízar,Dong Keon Yon,Bach Tran,Pascal Auquier,Guillaume Fond,Laurent Boyer

DOI: https://doi.org/10.2196/49916

2024-05-16

Abstract:Background: The care environment significantly influences the experiences of patients with severe mental illness and the quality of their care. While a welcoming and stimulating environment enhances patient satisfaction and health outcomes, psychiatric facilities often prioritize staff workflow over patient needs. Addressing these challenges is crucial to improving patient experiences and outcomes in mental health care. Objective: This study is part of the Patient-Reported Experience Measure for Improving Quality of Care in Mental Health (PREMIUM) project and aims to establish an item bank (PREMIUM-CE) and to develop computerized adaptive tests (CATs) to measure the experience of the care environment of adult patients with schizophrenia, bipolar disorder, or major depressive disorder. Methods: We performed psychometric analyses including assessments of item response theory (IRT) model assumptions, IRT model fit, differential item functioning (DIF), item bank validity, and CAT simulations. Results: In this multicenter cross-sectional study, 498 patients were recruited from outpatient and inpatient settings. The final PREMIUM-CE 13-item bank was sufficiently unidimensional (root mean square error of approximation=0.082, 95% CI 0.067-0.097; comparative fit index=0.974; Tucker-Lewis index=0.968) and showed an adequate fit to the IRT model (infit mean square statistic ranging between 0.7 and 1.0). DIF analysis revealed no item biases according to gender, health care settings, diagnosis, or mode of study participation. PREMIUM-CE scores correlated strongly with satisfaction measures (r=0.69-0.78; P<.001) and weakly with quality-of-life measures (r=0.11-0.21; P<.001). CAT simulations showed a strong correlation (r=0.98) between CAT scores and those of the full item bank, and around 79.5% (396/498) of the participants obtained a reliable score with the administration of an average of 7 items. Conclusions: The PREMIUM-CE item bank and its CAT version have shown excellent psychometric properties, making them reliable measures for evaluating the patient experience of the care environment among adults with severe mental illness in both outpatient and inpatient settings. These measures are a valuable addition to the existing landscape of patient experience assessment, capturing what truly matters to patients and enhancing the understanding of their care experiences. Trial registration: ClinicalTrials.gov NCT02491866; https://clinicaltrials.gov/study/NCT02491866.
Evidence-based assessment in clinical settings: Reducing assessment burden for a structured measure of child and adolescent anxiety.

Rebecca E. Ford-Paz,Karen R. Gouze,Caroline E. Kerns,Rachel Ballard,John T. Parkhurst,Poonam Jha,John Lavigne

DOI: https://doi.org/10.1037/ser0000367

2020-08-01

Psychological Services

Abstract:Clinically useful and evidence-based mental health assessment requires the identification of strategies that maximize diagnostic accuracy, inform treatment planning, and make efficient use of clinician and patient time and resources. This study uses classification tree analyses to determine whether parent- and child-report instruments, alone or in combination, can accurately predict diagnoses as measured by the Anxiety Disorders Interview Schedule (ADIS). The ADIS, which is the gold-standard semistructured interview for anxiety disorders in children and adolescents, requires formal training and lengthy administration. Data were collected as part of the standard diagnostic assessment process for 201 patients (ages 5 to 17 years) in an urban outpatient psychiatry specialty clinic. Analyses examined 2 models to determine which predictors reached an acceptable level of diagnostic accuracy for generalized anxiety, social anxiety, and separation anxiety disorders. The first model used scores on a parent- and child-report anxiety measure combined with demographic factors, and the second model incorporated a broad-band measure of child psychopathology and a depression measure into the analysis. Although demographic factors did not emerge as accurate predictors in either model, particular measures, either alone or in combination, were able to predict specific ADIS diagnoses in some cases, allowing for the potential streamlining of ADIS administration. These results suggest that a classification-tree analysis lends itself to the construction of simple algorithms that have high clinical utility and may advance the feasibility and utility of evidence-based assessment strategies in real-world practice settings by balancing cost effectiveness, administration demands, and accuracy. (PsycInfo Database Record (c) 2020 APA, all rights reserved)

psychology, clinical
Measurement precision and user experience with adaptive versus non-adaptive psychometric tests

Hanif Akhtar,Kristof Kovacs

DOI: https://doi.org/10.1016/j.paid.2024.112675

IF: 3.95

2024-04-18

Personality and Individual Differences

Abstract:Research on computerized adaptive testing (CAT) has provided evidence of the psychometric advantages of CAT over traditional fixed-item tests (FIT). However, evidence on test-taking experience is limited. This study investigated the psychometric and psychological impacts of CAT in the context of a multidimensional fluid reasoning test. In an online experimental study, we tested 286 participants, randomly assigning them to either the CAT or FIT conditions. We used two types of measures to assess participants' experience: self-report and time-based measures. The results showed that CAT outperforms FIT in terms of measurement precision but had a minimal impact on the test-taking experience. At the same time, FIT resulted in a more optimistic evaluation: when completing FIT, participants believed they had answered more items correctly than when completing CAT. Yet response time analysis indicates that participants invested more effort when working in CAT. Rapid guessing increased as the test progressed in both CAT and FIT, particularly in the FIT condition with the most difficult items. Providing information about how CAT works or starting the test with easier items may potentially enhance test-takers' experiences.

psychology, social
Examining cognitive disengagement syndrome in a psychiatric outpatient sample: Psychometric support and associations with internalizing symptoms and sleep problems

Bengu Yucens,Omer Basay,Ahmet Buber,Selim Tumkaya,Burge Kabukcu Basay,Busra Erdem,Stephen P. Becker,G. Leonard Burns

DOI: https://doi.org/10.1002/jclp.23678

2024-03-13

Journal of Clinical Psychology

Abstract:Objective The internal (structural) and external validity of a self‐report measure of cognitive disengagement syndrome (CDS, formerly sluggish cognitive tempo) relative to a self‐report measure of attention‐deficit/hyperactivity disorder‐inattention (ADHD‐IN) was evaluated with adults from university outpatient psychiatric clinics in Turkey. Methods A total of 274 outpatients (75.9% women; ages 18–64 years; Mage = 31.06; SDage = 10.84; 50.4% anxiety disorders; 41.6% depressive disorders; 2.9% ADHD; 1.5% sleep disorders; 0.7% eating disorders; 2.9% no mental disorder) completed self‐report measures of CDS, ADHD‐IN, ADHD‐hyperactivity/impulsivity (HI), sleep problems, depression, and stress. Results All 15 CDS symptoms measured by the Adult Concentration Inventory (ACI) showed convergent (moderate to high loadings on the CDS factor) and discriminant (loading close to zero on the ADHD‐IN factor) validity. CDS also showed stronger first‐order and unique associations than ADHD‐IN with sleep problems, depression, anxiety, and stress, whereas ADHD‐IN showed stronger first‐order and unique associations than CDS with ADHD‐HI. Conclusion This is the first study to provide support for the scores from this 15 item self‐report measure of CDS by the ACI in a clinical sample of adults, with findings consistent with previous studies examining parent and teacher rating scale measures with the same 15 CDS symptoms. These findings provide additional support for usefulness of these 15 CDS symptoms as measured by the ACI to study CDS across various cultures.

psychology, clinical
Methods for online calibration of Q-matrix and item parameters for polytomous responses in cognitive diagnostic computerized adaptive testing

Qingrong Tan,Daxun Wang,Fen Luo,Yan Cai,Dongbo Tu

DOI: https://doi.org/10.3758/s13428-024-02392-6

IF: 5.953

2024-05-02

Behavior Research Methods

Abstract:The ability to rapidly provide examinees with detailed and effective diagnostic information is a critical topic in psychology. Knowing what diagnostic criteria the examinees have met enables the practitioner to seek the solution to help them in a timely manner, and this can be achieved by cognitive diagnostic computerized adaptive testing (CD-CAT). However, the pervasive challenge of replenishing items in the CD-CAT item bank limits its practical application. Online calibration is a means to address item replenishment, but in CD-CAT, most existing online calibration methods that jointly calibrate the Q-matrix and item parameters of the new items are developed only for dichotomous responses and are time-consuming. Notably, previous studies pay no attention to polytomously scored items that are frequently observed in testing, even though they can offer additional evidence for the examinees' diagnosis. To fill this gap, we propose a SCAD-based method (SCAD-EM) to calibrate the Q-matrix and item parameters of the new items with polytomous response data in order to promote the application of CD-CAT in practice. The performance of the SCAD-EM was investigated in two comprehensive simulation studies and compared against the revised single-item estimation method (SIE-BIC). Results indicated that the SCAD-EM produces a higher calibration accuracy for the category-level Q-matrix and is computationally more efficient across all conditions, but it produces a lower calibration accuracy for the item-level Q-matrix. An empirical study further demonstrated the utility of the SCAD-EM and the SIE-BIC methods in calibrating new items with a real dataset. The advantages of the proposed method, its limitations, and possible future research directions are offered at the end.

psychology, mathematical, experimental
Cross-Validation and Development of Empirically Derived ADHD Assessment Strategies: Insights From the National Longitudinal Study of Adolescent Health (Add Health)

Shirag K. Shemmassian,Steve S. Lee

DOI: https://doi.org/10.1177/1087054717733042

2017-09-21

Journal of Attention Disorders

Abstract:Objective: There is replicated evidence that individual ADHD symptoms differentially predict ADHD diagnostic status, and that non– Diagnostic and Statistical Manual of Mental Disorders (DSM)-based symptom algorithms are superior to DSM criteria for ruling in or ruling out ADHD. However, these findings have been limited to case-control samples, despite the need to replicate them in independent and more representative samples and to increase their generalizability. Method: We analyzed the base rates and psychometric properties of ADHD symptoms in the population-based National Longitudinal Study of Adolescent Health (Add Health; http://www.cpc.unc.edu/projects/addhealth ) sample, and evaluated the predictive utility of empirically derived ADHD symptoms against the DSM approach with respect to academic, health, and relational functional outcomes. The sample consisted of 11,247 (54% female) ethnically diverse (45% non-White) individuals who retrospectively self-reported the severity of their ADHD symptoms from 5 to 12 years. Results: Individual ADHD symptoms variably predicted ADHD status, and whereas most symptoms were most predictive when endorsed at the two highest severity levels in previous studies, symptoms in Add Health were often most predictive of ADHD when endorsed at the highest severity level. Conclusion: The divergence in optimal severity thresholds for ADHD symptoms potentially reflects the different symptom base rates in clinical versus population-based samples. The accuracy of the different classification approaches also suggested that different symptom algorithms may be superior to evaluate ADHD depending on the assessment setting.

psychiatry,psychology, developmental
Psychometric properties of computerized adaptive testing for chronic obstructive pulmonary disease patient-reported outcome measurement

Jiajia Wang,Yang Xie,Zhenzhen Feng,Jiansheng Li

DOI: https://doi.org/10.1186/s12955-024-02291-6

2024-09-04

Abstract:Background: Computerized adaptive testing (CAT) is an effective way to reduce time, repetitious redundancy, and respond burden, and has been used to measure outcomes in many diseases. This study aimed to develop and validate a comprehensive disease-specific CAT for chronic obstructive pulmonary disease (COPD) patient-reported outcome measurement. Methods: The discrimination and difficulty of the items from the modified patient-reported outcome scale for COPD (mCOPD-PRO) were analyzed using item response theory. Then the initial item, item selection method, ability estimation method, and stopping criteria were further set based on Concerto platform to form the CAT. Finally, the reliability and validity were validated. Results: The item discrimination ranged from 1.05 to 2.71, and the item difficulty ranged from - 3.08 to 3.65. The measurement reliability of the CAT ranged from 0.910 to 0.922 using random method, while that ranged from 0.910 to 0.924 using maximum Fisher information (MFI) method. The content validity was good. The correlation coefficient between theta of the CAT and COPD assessment test and modified Medical Research Council dyspnea scale scores using random method was 0.628 and 0.540 (P < 0.001; P < 0.001) respectively, while that using MFI method was 0.347 and 0.328 (P = 0.007; P = 0.010) respectively. About 11 items (reducing by 59.3%) on average were tested using random method, while about seven items (reducing by 74.1%) on average using MFI method. The correlation coefficient between theta of the CAT and mCOPD-PRO total scores using random method was 0.919 (P < 0.001), while that using MFI method was 0.760 (P < 0.001). Conclusions: The comprehensive disease-specific CAT for COPD patient-reported outcome measurement is well developed with good psychometric properties, which can provide an efficient, accurate, and user-friendly measurement for patient-reported outcome of COPD.
Item response theory and computer adaptive testing of the sexual knowledge scale of the sexual knowledge and attitude test in a college sample

Martin Komarc,Aya Shigeto,Lawrence M. Scheier

DOI: https://doi.org/10.1080/19419899.2024.2332630

2024-03-23

Psychology and Sexuality

Abstract:Our study addresses the limited availability of well-validated scales for assessing general sexual knowledge among college populations. To fill this gap, we examined the psychometric properties of a 41-item sexual knowledge scale derived from the Sexual Knowledge and Attitudes Test – Adolescents (SKAT-A) in a sample of young adults aged 18–25 ( N = 1,291). We employed classical test theory (CTT) procedures, followed by item response theory (IRT) and computerized adaptive testing (CAT), to refine the SKAT-A knowledge scale. Both CTT and IRT analyses identified six items for removal due to poor discrimination and difficulty parameters. A confirmatory factor analysis supported a well-defined unidimensional latent trait of sexual knowledge. The results of CAT simulations using dynamic item administration demonstrated the scale's measurement precision, with moderate test reliability and relatively low standard errors. On average, there was a 54.3% reduction in the number of items administered without compromising scale reliability. This study concludes that the SKAT-A efficiently assesses a unidimensional trait of sexual knowledge in college-attending young adults. It highlights that only a subset of items from the full test bank is necessary to achieve this, providing a practical and reliable tool for assessing sexual knowledge in this population.

psychology, multidisciplinary
Convergent Validity of Experimental Cognitive Tests in a Large Community Sample

Andy C. Dean,Jean-Baptiste Pochon,Robert M. Bilder,Fred W. Sabb,Eliza Congdon,Dara Ghahremani,Katherine H. Karlsgodt,Theo G. M. van Erp,Rebecca F. Schwarzlose,Tyrone D. Cannon,Nelson B. Freimer,Edythe D. London

DOI: https://doi.org/10.1177/10731911241283410

IF: 4.282

2024-11-13

Assessment

Abstract:Assessment, Ahead of Print. Experimental cognitive tests are designed to measure particular cognitive domains, although evidence supporting test validity is often limited. The Consortium for Neuropsychiatric Phenomics test battery administered 23 experimental and traditional neuropsychological tests to a large sample of community volunteers (n = 1,059) and patients with psychiatric diagnoses (n = 137), providing a unique opportunity to examine convergent validity with factor analysis. Traditional tests included subtests from the Wechsler and Delis–Kaplan batteries, while experimental tests included the Attention Networks Test, Balloon Analogue Risk Task, Delay Discounting Task, Remember–Know, Reversal Learning Task, Scene Recognition, Spatial and Verbal Capacity and Manipulation Tasks, Stop-Signal Task, and Task Switching. Several experimental cognitive measures were insufficiently related to other tests and were excluded from factor analyses. In the remaining 18 tests, exploratory factor analysis and subsequent multigroup confirmatory factor analysis supported a three-factor structure broadly corresponding to domains of verbal/working memory, inhibitory control, and memory. In sum, several experimental measures of inhibitory control had weak relationships with all other tests, while the convergent validity of most tests of working memory and memory was supported.

psychology, clinical
Cross-validating the Clinical Assessment of Attention Deficit–Adult symptom validity scales for assessment of attention deficit/hyperactivity disorder in adults

John-Christopher A. Finley,Brian M. Cerny,Julia M. Brooks,Maximillian A. Obolsky,Aya Haneda,Gabriel P. Ovsiew,Devin M. Ulrich,Zachary J. Resch,Jason R. Soble

DOI: https://doi.org/10.1080/13803395.2023.2283940

2023-11-23

Journal of Clinical and Experimental Neuropsychology

Abstract:INTRODUCTION: The Clinical Assessment of Attention Deficit-Adult is among the few questionnaires that offer validity indicators (i.e., Negative Impression [NI], Infrequency [IF], and Positive Impression [PI]) for classifying underreporting and overreporting of attention-deficit/hyperactivity disorder (ADHD) symptoms. This is the first study to cross-validate the NI, IF, and PI scales in a sample of adults with suspected or known ADHD.METHOD: Univariate and multivariate analyses were conducted to examine the independent and combined value of the NI, IF, and PI scores in predicting invalid symptom reporting and neurocognitive performance in a sample of 543 adults undergoing ADHD evaluation.RESULTS: The NI scale demonstrated better classification accuracy than the IF scale in discriminating patients with and without valid scores on measures of overreporting. Only NI scores significantly predicted validity status when used in combination with IF scores. Optimal cut-scores for the NI (≤51; 30% sensitivity / 90% specificity) and IF (≥4; 18% sensitivity / 90% specificity) scales were consistent with those reported in the original manual; however, these indicators poorly discriminated patients with invalid and valid neurocognitive performance. The PI scale demonstrated acceptable classification accuracy in discriminating patients with invalid and valid scores on measures of underreporting, albeit with an optimal cut-score (≥27; 36% sensitivity / 90% specificity) lower than that described in the manual.CONCLUSION: Findings provide preliminary evidence of construct validity for these scales as embedded validity indicators of symptom overreporting and underreporting. However, these scales should not be used to guide clinical judgment regarding the validity of neurocognitive test performance.

psychology, clinical,clinical neurology

The interactive effects of nocturnal sleep and daytime naps in relation to serum C-reactive protein.

Developing an accurate and efficient tool for the internalizing spectrum: A simulation study of the adaptive algorithm to the Inventory of Depression and Anxiety Symptoms II (IDAS-II)

Validation of the structured interview section of the penn computerized adaptive test for neurocognitive and clinical psychopathology assessment (CAT GOASSESS)

Development and public release of a computerized adaptive (CAT) version of the Schizotypal Personality Questionnaire

Computerized Adaptive Testing for Sleep Disorders: Development of An Item Bank and Validation in A Simulated Study

Computerized adaptive testing to screen pre-school children for emotional and behavioral problems

Initial Validation of a Computerized Adaptive Test for Substance Use Disorder Identification in Adolescents

Developing Computerized Adaptive Testing for a National Health Professionals Exam: An Attempt from Psychometric Simulations

Efficiency of computerized adaptive testing with a cognitively designed item bank

Development of a Computerized Adaptive Testing for ADHD Using Bayesian Networks: an Attempt at Classification

The Comprehensive Autistic Trait Inventory (CATI): development and validation of a new measure of autistic traits in the general population

Psychometric Assessment of an Item Bank for Adaptive Testing on Patient-Reported Experience of Care Environment for Severe Mental Illness: Validation Study

Evidence-based assessment in clinical settings: Reducing assessment burden for a structured measure of child and adolescent anxiety.

Measurement precision and user experience with adaptive versus non-adaptive psychometric tests

Examining cognitive disengagement syndrome in a psychiatric outpatient sample: Psychometric support and associations with internalizing symptoms and sleep problems

Methods for online calibration of Q-matrix and item parameters for polytomous responses in cognitive diagnostic computerized adaptive testing

Cross-Validation and Development of Empirically Derived ADHD Assessment Strategies: Insights From the National Longitudinal Study of Adolescent Health (Add Health)

Psychometric properties of computerized adaptive testing for chronic obstructive pulmonary disease patient-reported outcome measurement

Item response theory and computer adaptive testing of the sexual knowledge scale of the sexual knowledge and attitude test in a college sample

Convergent Validity of Experimental Cognitive Tests in a Large Community Sample

Cross-validating the Clinical Assessment of Attention Deficit–Adult symptom validity scales for assessment of attention deficit/hyperactivity disorder in adults