Neuropsychiatric Disease Classification Using Functional Connectomics -- Results of the Connectomics in NeuroImaging Transfer Learning Challenge

Markus D. Schirmer,Archana Venkataraman,Islem Rekik,Minjeong Kim,Stewart H. Mostofsky,Mary Beth Nebel,Keri Rosch,Karen Seymour,Deana Crocetti,Hassna Irzan,Michael Hütel,Sebastien Ourselin,Neil Marlow,Andrew Melbourne,Egor Levchenko,Shuo Zhou,Mwiza Kunda,Haiping Lu,Nicha C. Dvornek,Juntang Zhuang,Gideon Pinto,Sandip Samal,Jennings Zhang,Jorge L. Bernal-Rusiel,Rudolph Pienaar,Ai Wern Chung
DOI: https://doi.org/10.48550/arXiv.2006.03611
2020-11-25
Abstract:Large, open-source consortium datasets have spurred the development of new and increasingly powerful machine learning approaches in brain connectomics. However, one key question remains: are we capturing biologically relevant and generalizable information about the brain, or are we simply overfitting to the data? To answer this, we organized a scientific challenge, the Connectomics in NeuroImaging Transfer Learning Challenge (CNI-TLC), held in conjunction with MICCAI 2019. CNI-TLC included two classification tasks: (1) diagnosis of Attention-Deficit/Hyperactivity Disorder (ADHD) within a pre-adolescent cohort; and (2) transference of the ADHD model to a related cohort of Autism Spectrum Disorder (ASD) patients with an ADHD comorbidity. In total, 240 resting-state fMRI time series averaged according to three standard parcellation atlases, along with clinical diagnosis, were released for training and validation (120 neurotypical controls and 120 ADHD). We also provided demographic information of age, sex, IQ, and handedness. A second set of 100 subjects (50 neurotypical controls, 25 ADHD, and 25 ASD with ADHD comorbidity) was used for testing. Models were submitted in a standardized format as Docker images through ChRIS, an open-source image analysis platform. Utilizing an inclusive approach, we ranked the methods based on 16 different metrics. The final rank was calculated using the rank product for each participant across all measures. Furthermore, we assessed the calibration curves of each method. Five participants submitted their model for evaluation, with one outperforming all other methods in both ADHD and ASD classification. However, further improvements are needed to reach the clinical translation of functional connectomics. We are keeping the CNI-TLC open as a publicly available resource for developing and validating new classification methodologies in the field of connectomics.
Neurons and Cognition,Machine Learning
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is to evaluate the transfer learning ability of functional connectomics in neuroimaging, especially for the classification tasks of attention - deficit/hyperactivity disorder (ADHD) and autism spectrum disorder (ASD). Specifically, the researchers organized a scientific challenge - Connectomics in NeuroImaging Transfer Learning Challenge (CNI - TLC), aiming to explore the following two questions: 1. **Diagnostic accuracy**: Is it possible to accurately distinguish ADHD patients from the normal control group by using resting - state functional magnetic resonance imaging (rsfMRI) data and clinical diagnosis information? 2. **Generalization ability of the model**: Can the trained ADHD classification model be effectively transferred to another ASD patient group with ADHD comorbidity, that is, whether the feature representation of the model has the generalization ability across diseases? To answer these questions, CNI - TLC designed two classification tasks: - **Task 1**: Diagnose ADHD in a pre - adolescent cohort. - **Task 2**: Transfer the ADHD model to a related ASD patient cohort where these patients also have ADHD. The researchers provided 240 resting - state fMRI time - series data, which were averaged according to three standard parcellation atlases, and also provided demographic information such as age, gender, IQ and handedness. In addition, they used 16 different evaluation metrics to comprehensively evaluate the performance of each submitted method, including accuracy, area under the curve (AUC), F1 - score, etc. Finally, through the comprehensive evaluation of these metrics, the researchers hope to find the methods that can perform best in ADHD and ASD classification tasks and explore the biological relevance and generalization ability of these methods.