Abstract:Background: Early detection of mild cognitive impairment (MCI), a transitional stage between normal aging and Alzheimer disease, is crucial for preventing the progression of dementia. Virtual reality (VR) biomarkers have proven to be effective in capturing behaviors associated with subtle deficits in instrumental activities of daily living, such as challenges in using a food-ordering kiosk, for early detection of MCI. On the other hand, magnetic resonance imaging (MRI) biomarkers have demonstrated their efficacy in quantifying observable structural brain changes that can aid in early MCI detection. Nevertheless, the relationship between VR-derived and MRI biomarkers remains an open question. In this context, we explored the integration of VR-derived and MRI biomarkers to enhance early MCI detection through a multimodal learning approach. Objective: We aimed to evaluate and compare the efficacy of VR-derived and MRI biomarkers in the classification of MCI while also examining the strengths and weaknesses of each approach. Furthermore, we focused on improving early MCI detection by leveraging multimodal learning to integrate VR-derived and MRI biomarkers. Methods: The study encompassed a total of 54 participants, comprising 22 (41%) healthy controls and 32 (59%) patients with MCI. Participants completed a virtual kiosk test to collect 4 VR-derived biomarkers (hand movement speed, scanpath length, time to completion, and the number of errors), and T1-weighted MRI scans were performed to collect 22 MRI biomarkers from both hemispheres. Analyses of covariance were used to compare these biomarkers between healthy controls and patients with MCI, with age considered as a covariate. Subsequently, the biomarkers that exhibited significant differences between the 2 groups were used to train and validate a multimodal learning model aimed at early screening for patients with MCI among healthy controls. Results: The support vector machine (SVM) using only VR-derived biomarkers achieved a sensitivity of 87.5% and specificity of 90%, whereas the MRI biomarkers showed a sensitivity of 90.9% and specificity of 71.4%. Moreover, a correlation analysis revealed a significant association between MRI-observed brain atrophy and impaired performance in instrumental activities of daily living in the VR environment. Notably, the integration of both VR-derived and MRI biomarkers into a multimodal SVM model yielded superior results compared to unimodal SVM models, achieving higher accuracy (94.4%), sensitivity (100%), specificity (90.9%), precision (87.5%), and F1-score (93.3%). Conclusions: The results indicate that VR-derived biomarkers, characterized by their high specificity, can be valuable as a robust, early screening tool for MCI in a broader older adult population. On the other hand, MRI biomarkers, known for their high sensitivity, excel at confirming the presence of MCI. Moreover, the multimodal learning approach introduced in our study provides valuable insights into the improvement of early MCI detection by integrating a diverse set of biomarkers.

A portable and efficient dementia screening tool using eye tracking machine learning and virtual reality

Digitized Biomarkers Utilizing Human‐Computer Interaction Sensing Technology for Early Auxiliary Diagnosis of Alzheimer’s Disease

Screening for Mild Cognitive Impairment with Speech Interaction Based on Virtual Reality and Wearable Devices

Digital Marker for Early Screening of Mild Cognitive Impairment Through Hand and Eye Movement Analysis in Virtual Reality Using Machine Learning: First Validation Study

Develop a Diagnostic Tool for Dementia Using Machine Learning and Non-Imaging Features

A Pilot Tool of the Virtual Scenario Initial Dementia Cognitive Screening (VSIDCS) with a Cultural Exhibition for Improving the Standard Traditional Test

VRNPT: A Neuropsychological Test Tool for Diagnosing Mild Cognitive Impairment Using Virtual Reality and EEG Signals

Detection of Mild Cognitive Impairment Based on Virtual Reality: A Scoping Review

A Virtual Supermarket Program for the Screening of Mild Cognitive Impairment in Older Adults: Diagnostic Accuracy Study

A Novel Detection Tool for Mild Cognitive Impairment Patients Based on Eye Movement and Electroencephalogram

Exploratory Research on Key Technology of Human-Computer Interactive 2.5-Minute Fast Digital Early Warning for Mild Cognitive Impairment

Early Diagnosis of Mild Cognitive Impairment Based on Eye Movement Parameters in an Aging Chinese Population

Unearthing Subtle Cognitive Variations: A Digital Screening Tool for Detecting and Monitoring Mild Cognitive Impairment

An Ensemble Learning Algorithm for Cognitive Evaluation by an Immersive Virtual Reality Supermarket

Diagnosing and tracking depression based on eye movement in response to virtual reality

Integrating Biomarkers From Virtual Reality and Magnetic Resonance Imaging for the Early Detection of Mild Cognitive Impairment Using a Multimodal Learning Approach: Validation Study

Eye-tracking-based cognitive assessment efficiently detects mild cognitive decline in the predementia stage

Validation and comparison of virtual reality and 3D mobile games for cognitive assessment against ACE-III in 82 young participants

Can a virtual reality cognitive training application fulfill a dual role? Using the virtual supermarket cognitive training application as a screening tool for mild cognitive impairment

Visual Cognitive Assessment Test: Utility of the brief cognitive battery for early screening of cognitive impairment in Chongqing, China

Automated Non‐Supervised Eye Disorder Screening System Using Virtual Reality