Abstract:Background: Early detection of mild cognitive impairment (MCI), a transitional stage between normal aging and Alzheimer disease, is crucial for preventing the progression of dementia. Virtual reality (VR) biomarkers have proven to be effective in capturing behaviors associated with subtle deficits in instrumental activities of daily living, such as challenges in using a food-ordering kiosk, for early detection of MCI. On the other hand, magnetic resonance imaging (MRI) biomarkers have demonstrated their efficacy in quantifying observable structural brain changes that can aid in early MCI detection. Nevertheless, the relationship between VR-derived and MRI biomarkers remains an open question. In this context, we explored the integration of VR-derived and MRI biomarkers to enhance early MCI detection through a multimodal learning approach. Objective: We aimed to evaluate and compare the efficacy of VR-derived and MRI biomarkers in the classification of MCI while also examining the strengths and weaknesses of each approach. Furthermore, we focused on improving early MCI detection by leveraging multimodal learning to integrate VR-derived and MRI biomarkers. Methods: The study encompassed a total of 54 participants, comprising 22 (41%) healthy controls and 32 (59%) patients with MCI. Participants completed a virtual kiosk test to collect 4 VR-derived biomarkers (hand movement speed, scanpath length, time to completion, and the number of errors), and T1-weighted MRI scans were performed to collect 22 MRI biomarkers from both hemispheres. Analyses of covariance were used to compare these biomarkers between healthy controls and patients with MCI, with age considered as a covariate. Subsequently, the biomarkers that exhibited significant differences between the 2 groups were used to train and validate a multimodal learning model aimed at early screening for patients with MCI among healthy controls. Results: The support vector machine (SVM) using only VR-derived biomarkers achieved a sensitivity of 87.5% and specificity of 90%, whereas the MRI biomarkers showed a sensitivity of 90.9% and specificity of 71.4%. Moreover, a correlation analysis revealed a significant association between MRI-observed brain atrophy and impaired performance in instrumental activities of daily living in the VR environment. Notably, the integration of both VR-derived and MRI biomarkers into a multimodal SVM model yielded superior results compared to unimodal SVM models, achieving higher accuracy (94.4%), sensitivity (100%), specificity (90.9%), precision (87.5%), and F1-score (93.3%). Conclusions: The results indicate that VR-derived biomarkers, characterized by their high specificity, can be valuable as a robust, early screening tool for MCI in a broader older adult population. On the other hand, MRI biomarkers, known for their high sensitivity, excel at confirming the presence of MCI. Moreover, the multimodal learning approach introduced in our study provides valuable insights into the improvement of early MCI detection by integrating a diverse set of biomarkers.

Screening for Mild Cognitive Impairment with Speech Interaction Based on Virtual Reality and Wearable Devices

VRNPT: A Neuropsychological Test Tool for Diagnosing Mild Cognitive Impairment Using Virtual Reality and EEG Signals

Synergy through integration of digital cognitive tests and wearable devices for mild cognitive impairment screening

Detection of Mild Cognitive Impairment Based on Virtual Reality: A Scoping Review

Integrating Biomarkers From Virtual Reality and Magnetic Resonance Imaging for the Early Detection of Mild Cognitive Impairment Using a Multimodal Learning Approach: Validation Study

A portable and efficient dementia screening tool using eye tracking machine learning and virtual reality

Virtual reality technology in the detection of mild cognitive impairment: A systematic review and meta-analysis

A multimodal cross-transformer-based model to predict mild cognitive impairment using speech, language and vision

A Novel Detection Tool for Mild Cognitive Impairment Patients Based on Eye Movement and Electroencephalogram

Mild Cognitive Impairment detection based on EEG and HRV data

Unearthing Subtle Cognitive Variations: A Digital Screening Tool for Detecting and Monitoring Mild Cognitive Impairment

An Ensemble Learning Algorithm for Cognitive Evaluation by an Immersive Virtual Reality Supermarket

Classification of Mild Cognitive Impairment Based on Handwriting Dynamics and Qeeg

Digital Biomarkers for the Early Detection of Mild Cognitive Impairment: Artificial Intelligence Meets Virtual Reality

A Virtual Supermarket Program for the Screening of Mild Cognitive Impairment in Older Adults: Diagnostic Accuracy Study

Exploiting Longitudinal Speech Sessions via Voice Assistant Systems for Early Detection of Cognitive Decline

Exploratory Research on Key Technology of Human-Computer Interactive 2.5-Minute Fast Digital Early Warning for Mild Cognitive Impairment

Detection of Mild Cognitive Impairment Through Natural Language and Touchscreen Typing Processing

The Impact of Virtual Reality-Based Products on Mild Cognitive Impairment Senior Subjects: an Experimental Study Using Multiple Sources of Data

CogniVoice: Multimodal and Multilingual Fusion Networks for Mild Cognitive Impairment Assessment from Spontaneous Speech

Selecting and Analyzing Speech Features for the Screening of Mild Cognitive Impairment