Abstract:This study uses a new method to summarize diagnostic validity information to explore which constructs are captured by malingering tests. The Test Validation Summary applies mixed-groups validation to investigate the meaning of test constructs and to estimate test classification characteristics when test validation groups are not "pure" criterion groups (ie, "compliant" vs "malingering"), but members have variable probability of malingering. The method permits the use of tests with relatively low validity to validate tests of greater validity. In our initial analysis, we argue that the Rey 15-Item Memory Test is best construed as an "intention test" (capturing the intention of testtakers when taking a test) as opposed to an "effort test." Using the Test Validation Summary and mixed-groups validation, we demonstrate that as an indicator of "intention to feign cognitive impairment," the Rey 15-Item Memory Test has estimated false-positive rate (FPR) = 0.02 and true-positive rate (TPR) = 0.57. We then explore the meaning of failure on the Word Memory Test (WMT), which uses a dichotomous classification of performance as valid or invalid. Although the WMT is commonly referred to as an "effort test," we argue that it likely captures both "intention" and "effort" but collapses this information into a single dichotomous classification of symptom validity. We demonstrate that, as a result of this dichotomous classification process, the WMT likely has a problematic FPR. In our analysis of previously published WMT data, the WMT FPR is estimated at 0.12 when there is no predisposition to perform poorly but rises dramatically and unrealistically as the predisposition to perform poorly increases. We compare these findings to those of the Validity Indicator Profile (VIP), which captures both intent and effort to classify 4 different sorts of response styles in cognitive testing. In our analyses, the VIP demonstrates that FPR = 0 and TPR = 0.86 when the construct being measured is "intent to perform poorly," and reveals that FPR = 0.06 and TPR = 0.63 when the construct being measured is "inconsistent responding" or "poor effort." We were able to demonstrate for the VIP the same "oversensitivity" shown by the WMT when the VIP was interpreted only as a dichotomous classification test. These results indicate that researchers who attempt to generate classification characteristics for malingering tests must carefully consider what constructs are being captured by the test.

Validity of the Groningen Effort Test in patients with suspected chronic solvent-induced encephalopathy

Further examination of embedded performance validity indicators for the Conners’ Continuous Performance Test and Brief Test of Attention in a large outpatient clinical sample

Assessing performance validity during attention-deficit/hyperactivity disorder evaluations: Cross-validation of non-memory embedded validity indicators

Detecting malingered neurocognitive dysfunction: Comparative analysis of freestanding and embedded performance validity tests

Two validation studies of a performance validity test for autistic adults

Performance validity in a presurgical epilepsy population

Psychometric implications of failure on one performance validity test: a cross-validation study to inform criterion group definition

Performance and symptom validity indicators among children undergoing cognitive surveillance following treatment for craniopharyngioma

Cross-Validation of Supplemental Test of Memory Malingering Scores as Performance Validity Measures

Assessment of differential neurocognitive performance based on the number of performance validity tests failures: A cross-validation study across multiple mixed clinical samples

Evaluating constructs represented by symptom validity tests in forensic neuropsychological assessment of traumatic brain injury

Criterion validity and sensitivity to change of the Early Rehabilitation Index (ERI): results from a German multi-center study

Use of perceptual memory as a performance validity indicator: initial validation with simulated mild traumatic brain injury

Validity of Web-based, Self-directed, NeuroCognitive Performance Test in MCI

An Examination of WAIS-IV Digit Span Performance Inconsistency as a Novel Embedded Performance Validity Test Among Adults Clinically Referred for Attention-Deficit/Hyperactivity Disorder

Methods for assessment of rey auditory verbal learning test performance in memory clinic patients and healthy adults - at the cross-roads of learning theory and clinical utility

A sentence completion procedure as an alternative to the Autobiographical Memory Test for assessing overgeneral memory in non-clinical populations

Comparison of six tests for diagnosing minimal hepatic encephalopathy and predicting clinical outcome – A prospective, observational study

What are the chances? – Validity assessment and the cumulative binomial distribution in the evaluation of fitness to stand trial

Validity of the brief executive-function assessment tool in an outpatient substance use disorder setting