Abstract:Objective: Recent consensus guidelines have advocated for the use of multivariate performance validity assessment on ability-based measures such those used in neuropsychological assessment. Further, previous research has demonstrated that aggregating performance validity indicators may produce superior classification accuracy. The present study builds upon this research by aggregating data from three of the most commonly used performance validity measures (Test of Memory Malingering [TOMM], Rey Fifteen Item Test with recognition trial [FIT plus recognition], and Reliable Digit Span [RDS]) to create a performance validity composite measure in a veteran mild traumatic brain injury (mTBI) population. Participants and Methods: Data of patients evaluated at a VA hospital who had completed the RDS, FIT plus recognition, and TOMM as part of their clinical neuropsychological evaluation were analyzed (n = 20). Two composite performance validity indexes were created: a Single Cutoff Performance Validity Index (SC-PVI), which measures the quantity of failures across performance validity measures (PVMs) by summing the total number of PVM failures, and a Multiple Cutoff Performance Validity Index (MC-PVI) which measures the number of failures as well as degree of failure(s) across measures of performance validity (e.g., a participant would attain a score of 3 if their PVM performance failed to reach a conservative cut point; they would obtain a score of 1 if they met conservative cut point, yet failed to reach a liberal cut point). Results: Only one participant (5%) attained a score of 0 on the SC-PVI (i.e., passing all PVTs using standard cutoffs) and MC-PVI (i.e., passing the most liberal cut points on all three PVMs). Conversely, eight participants (40%) attained a score of 3 on the SC-PVI (i.e., failed all three PVMs) and four participants (20%) attained a score of 9 (i.e., failed the most conservative cut points on all three PVMs). Results showed a significant (p < .001) ordinal association between the two indices (G = .984); however, there was no significant agreement between SC-PVI and MC-PVI models (k = -.087; p = .127). Conclusions: Data revealed discordant findings between the three PVMs utilized. The majority of participants (75%) scored between 2-8 on the MC-PVI, meaning that they did not exceed all liberal cut points or fail all conservative cut points. These "grey area" scores suggest an indeterminate range of performance validity, which cannot be captured by a solitary cut point or neatly classified as pass or fail. The utility of multiple cutoff performance validity models (i.e., aggregating PVMs to consider the severity of failure and number of failures) is that they capture the nuance of these data when determining and discussing the credibility of a profile. Multiple cut point data also highlight how the choice of cutoff influences the outcome of performance validity research and clinical decision making. As such, future research on the classification accuracy of this MC-PVI is needed.

That old dog can still hunt—Alternative cutoffs and the recognition trial improve the classification accuracy of the Rey 15-Item Test.

The classification accuracy of Warrington's recognition memory test (words) as a performance validity Test in a neurorehabilitation setting

Recognition subtests for the Repeatable Battery for the Assessment of Neuropsychological Status: Preliminary data in cognitively intact older adults, amnestic Mild Cognitive Impairment, and Alzheimer's disease

Recognition Subtests for Form B of the Repeatable Battery for the Assessment of Neuropsychological Status: Preliminary Validation

Discriminant Validity and Time‐effectiveness of A Stepwise Dementia Case‐finding Approach in an Asian Elderly Community

A - 07 Three Brief, Public-Domain Performance-Validity Measures for Concussion Baseline Testing: Normative Performance and Sub-Optimal Cut Scores

Further examination of embedded performance validity indicators for the Conners’ Continuous Performance Test and Brief Test of Attention in a large outpatient clinical sample

Methods for assessment of rey auditory verbal learning test performance in memory clinic patients and healthy adults - at the cross-roads of learning theory and clinical utility

81 An Examination of a Multiple Cutoff Performance Validity Index

MACE for Diagnosis of Dementia and MCI: Examining Cut-Offs and Predictive Values

Analyzing the relationship between processing speed impairment and Rey-15 item test performance

Psychometric implications of failure on one performance validity test: a cross-validation study to inform criterion group definition

Assessing performance validity during attention-deficit/hyperactivity disorder evaluations: Cross-validation of non-memory embedded validity indicators

On the Clinimetrics of the Montreal Cognitive Assessment: Cutoff Analysis in Patients with Mild Cognitive Impairment due to Alzheimer's Disease

Ten-words recall test: an effective tool to differentiate mild cognitive impairment from subjective cognitive decline

Comparative Study of Discerning Efficacy of Shortened Neuropsychological Tests on Mild Cognitive Impairment

UCancellation: A new mobile measure of selective attention and concentration

Improving the Measurement of Cognitive Ability in Geriatric Patients

Re‐examining the impact of adjusting for education and sex the RBANS delayed memory score: Implications on the cut‐offs used to detect cognitive impairment in prodromal Alzheimer’s disease. Data from the EPAD‐LCS study.

Comparative Performance of Five Cognitive Screening Tests in a Large Sample of Seniors