Abstract:Background: The Vestibular/Ocular-Motor Screening (VOMS) is a valuable component of acute (<72 hours) sports-related concussion (SRC) assessments and is increasingly used with the Immediate Post-concussion Assessment and Cognitive Testing (ImPACT) instrument and the third edition of the Sport Concussion Assessment Tool (SCAT3). Research has suggested that VOMS acute postinjury scores are useful in identifying acute concussion. However, the utility of preseason baseline measurements to improve diagnostic accuracy remains ambiguous. To this end, there is a need to determine how reliable VOMS baseline assessments are across years and whether incorporating individuals' baseline performance improves diagnostic yield for acute concussions. Purpose: To analyze VOMS, SCAT3, and ImPACT to evaluate the test-retest reliability of consecutive-year preseason baseline assessments to directly compare the diagnostic utility of these tools when incorporating baseline assessments versus using postinjury data alone to identify acute SRC. Study design: Cohort study (diagnosis); Level of evidence, 2. Methods: Preseason and postinjury VOMS, SCAT3, ImPACT Post-Concussion Symptom Scale (PCSS), and ImPACT composite scores were analyzed for 3958 preseason (47.7% female) and 496 acute (≤48 hours) SRC (37.5% female) collegiate athlete evaluations in the National Collegiate Athletic Association-Department of Defense Concussion Assessment Research and Education Consortium. Descriptive statistics, Kolmogorov-Smirnov significance, and Cohen d effect size were calculated. Consecutive-year baseline reliability was evaluated for a subset of 447 athlete encounters using Pearson r, Cohen κ, Cohen d, and 2-way mixed intraclass correlation coefficients (ICCs). Wilcoxon signed rank tests were used to determine the statistical significance between population performances, and the 90% reliable change index (RCI) was calculated from the test-retest results. Preseason to postinjury change scores were then calculated from each tool's RCI. Finally, receiver operating characteristic (ROC) curve analyses were conducted, and DeLong method was used to compare the area under the curve (AUC) of raw postinjury scores versus change scores from preseason baseline assessments. Potential effects of sex, medical history (learning disorders or attention-deficit/hyperactivity disorder), and outlier data were also explored. Results: Effect sizes were large, and overall predictive utilities were clinically useful for postinjury VOMS Total (d = 2.44; AUC = 0.85), the SCAT3 Symptom Evaluation total severity score (d = 1.74; AUC = 0.82), and the ImPACT PCSS total severity score (d = 1.67; AUC = 0.80). Comparatively, effect sizes were small and predictive utilities were poor for Standardized Assessment of Concussion (SAC), modified Balance Error Scoring System (mBESS), and all ImPACT composites (d = 0.11-0.46; AUC = 0.48-0.59). Preseason baseline test-retest reliability was poor to moderate (r = 0.23-0.52; κ = 0.32-0.36; ICC = 0.36-0.68) for all assessments except ImPACT Visual Motion Sensitivity (r = 0.73; ICC = 0.85). Incorporating baseline scores for VOMS Total, SCAT3 (Symptom Evaluation, SAC, mBESS), ImPACT PCSS, or ImPACT composites did not significantly improve AUCs. Conclusion: VOMS Total and symptom severity (SCAT3, PCSS) total scores had large effect sizes and clinically useful AUCs for identifying acute concussion. However, all tools demonstrated high within-patient test-retest variability, resulting in poor reliability. The findings in this sample of collegiate athletes suggest that incorporating baseline assessments does not significantly increase diagnostic yield for acute concussion.

A - 07 Three Brief, Public-Domain Performance-Validity Measures for Concussion Baseline Testing: Normative Performance and Sub-Optimal Cut Scores

81 An Examination of a Multiple Cutoff Performance Validity Index

Psychometric implications of failure on one performance validity test: a cross-validation study to inform criterion group definition

Further examination of embedded performance validity indicators for the Conners’ Continuous Performance Test and Brief Test of Attention in a large outpatient clinical sample

Baseline Normative and Test-Retest Reliability Data for Sideline Concussion Assessment Measures in Youth

Minimal Detectable Change for the ImPACT Subtests at Baseline

Cross-Validation of Supplemental Test of Memory Malingering Scores as Performance Validity Measures

Utility of VOMS, SCAT3, and ImPACT Baseline Evaluations for Acute Concussion Identification in Collegiate Athletes: Findings From the NCAA-DoD Concussion Assessment, Research and Education (CARE) Consortium

Assessing performance validity during attention-deficit/hyperactivity disorder evaluations: Cross-validation of non-memory embedded validity indicators

Detecting malingered neurocognitive dysfunction: Comparative analysis of freestanding and embedded performance validity tests

The prevalence and correlates of scores falling below ImPACT embedded validity indicators among adolescent student athletes

Assessment of differential neurocognitive performance based on the number of performance validity tests failures: A cross-validation study across multiple mixed clinical samples

A - 22 Baseline testing for athletes: where are we now and where are we going?

A comparison of embedded validity indicators from the Stroop Color and Word Test among adults referred for clinical evaluation of suspected or confirmed attention-deficit/hyperactivity disorder.

Comparison of Urban versus Industry Normative Values of Immediate Post-Concussion Assessment and Cognitive Testing (ImPACTTM)

Use of perceptual memory as a performance validity indicator: initial validation with simulated mild traumatic brain injury

85 Performance Consistency on a Measure of Sustained and Selective Attention

Introduction to the special issue on performance validity: what are we doing? What should we do?

Test Performance and Test-Retest Reliability of the Vestibular/Ocular Motor Screening and King-Devick Test in Adolescent Athletes During a Competitive Sport Season

Measures of performance validity for Hispanic/Latin American adults: A PRISMA systematic review.