Testing the Tests: Using Connectome-Based Predictive Models to Reveal the Systems Standardized Tests and Clinical Symptoms Are Reflecting

Anja Samardzija,Xilin Shen,Wenjing Luo,Abigail Sara Greene,Saloni Mehta,Fuyuze Tokoglu,Jagriti Arora,Scott Woods,Rachel Katz,Gerard Sanacora,Vinod H. Srihari,Dustin Scheinost,Robert T Constable

DOI: https://doi.org/10.1101/2024.10.24.619737

2024-10-26

Abstract:Neuroimaging has achieved considerable success in elucidating the neurophysiological underpinnings of various brain functions. Tools such as standardized cognitive tests and symptom inventories have played a crucial role in informing neuroimaging studies, helping to uncover the underlying brain systems associated with these measures. Substantial strides have been taken in developing models, such as connectome-based predictive modeling (CPM), that establish connections between external measures and the human connectome, offering insights into how the functional organization of the brain varies in relation to scores on external measures. Here, we depart from the conventional feed-forward approach and introduce a feed-back approach that allows testing of the tests. Since the inception of cognitive psychology over 60 years ago, cognitive tests have been meticulously developed to measure specific components of cognition. These tests, which have undergone extensive validation and have been standardized and administered to millions, operate on explicit assumptions about the cognitive components they assess. Rather than using external tests to identify the circuits supporting test scores, we a priori define networks of interest and quantify the extent to which these circuits support the test measure. To demonstrate this, we define functional connectivity networks for six cognitive constructs and quantify their contribution to performance across a spectrum of standardized cognitive tests and clinical measures. Employing robust machine learning in a predictive modeling framework, we show how this approach can be used to select tests according to the networks they rely upon. This establishes a biologically grounded metric for test comparison. This approach also yields a brain-driven process for forming composite tests by selecting test combinations that depend on the same proportional brain systems, or for a single network of interest, combining tests with the highest predictive power for that network. This brain-driven approach results in more robust behavioral assessments and enhanced predictive power for the network of interest. We illustrate how this methodology can be applied to evaluate the inclusion of specific sub-tests within a composite score, revealing instances where composite scores are reinforced or weakened by subtest inclusion in terms of the specificity of the brain network they interrogate. The brain-test score modeling approach presented here provides a biologically driven approach to the selection of external cognitive and symptom measures directed at specific brain systems. It opens new avenues of research by providing a framework for the development of new tests and measures guided by quantitative brain metrics.

Neuroscience