Abstract:Background: With the rapid aging of the global population, the prevalence of mild cognitive impairment (MCI) and dementia is anticipated to surge worldwide. MCI serves as an intermediary stage between normal aging and dementia, necessitating more sensitive and effective screening tools for early identification and intervention. The BrainFx SCREEN is a novel digital tool designed to assess cognitive impairment. This study evaluated its efficacy as a screening tool for MCI in primary care settings, particularly in the context of an aging population and the growing integration of digital health solutions. Objective: The primary objective was to assess the validity, reliability, and applicability of the BrainFx SCREEN (hereafter, the SCREEN) for MCI screening in a primary care context. We conducted an exploratory study comparing the SCREEN with an established screening tool, the Quick Mild Cognitive Impairment (Qmci) screen. Methods: A concurrent mixed methods, prospective study using a quasi-experimental design was conducted with 147 participants from 5 primary care Family Health Teams (FHTs; characterized by multidisciplinary practice and capitated funding) across southwestern Ontario, Canada. Participants included health care practitioners, patients, and FHT administrative executives. Individuals aged ≥55 years with no history of MCI or diagnosis of dementia rostered in a participating FHT were eligible to participate. Participants were screened using both the SCREEN and Qmci. The study also incorporated the Geriatric Anxiety Scale–10 to assess general anxiety levels at each cognitive screening. The SCREEN's scoring was compared against that of the Qmci and the clinical judgment of health care professionals. Statistical analyses included sensitivity, specificity, internal consistency, and test-retest reliability assessments. Results: The study found that the SCREEN's longer administration time and complex scoring algorithm, which is proprietary and unavailable for independent analysis, presented challenges. Its internal consistency, indicated by a Cronbach α of 0.63, was below the acceptable threshold. The test-retest reliability also showed limitations, with moderate intraclass correlation coefficient (0.54) and inadequate κ (0.15) values. Sensitivity and specificity were consistent (63.25% and 74.07%, respectively) between cross-tabulation and discrepant analysis. In addition, the study faced limitations due to its demographic skew (96/147, 65.3% female, well-educated participants), the absence of a comprehensive gold standard for MCI diagnosis, and financial constraints limiting the inclusion of confirmatory neuropsychological testing. Conclusions: The SCREEN, in its current form, does not meet the necessary criteria for an optimal MCI screening tool in primary care settings, primarily due to its longer administration time and lower reliability. As the number of digital health technologies increases and evolves, further testing and refinement of tools such as the SCREEN are essential to ensure their efficacy and reliability in real-world clinical settings. This study advocates for continued research in this rapidly advancing field to better serve the aging population. Trial Registration:

Validating cognitive screening in young people with first‐episode psychosis: The CogScreen protocol

The development of a cognitive screening protocol for Aboriginal and/or Torres Strait Islander peoples: the Guddi Way screen

Validation of the structured interview section of the penn computerized adaptive test for neurocognitive and clinical psychopathology assessment (CAT GOASSESS)

Psychometric Evaluation of a Tablet-Based Tool to Detect Mild Cognitive Impairment in Older Adults: Mixed Methods Study

Protocol for a construct and clinical validation study of MyCog Mobile: a remote smartphone-based cognitive screener for older adults

Acceptability, engagement, and efficacy of cognitive remediation for cognitive outcomes in young adults with first-episode psychosis and social anxiety: A randomized-controlled trial

Cognitive Remediation and Social Recovery in Early Psychosis (CReSt-R): protocol for a pilot randomised controlled study

Cleveland Clinic Cognitive Battery (C3B): Normative, Reliability, and Validation Studies of a Self-Administered Computerized Tool for Screening Cognitive Dysfunction in Primary Care

Don't be late! Timely identification of cognitive impairment in people with multiple sclerosis: a study protocol

Youth Mental Health Tracker: protocol to establish a longitudinal cohort and research database for young people attending Australian mental health services

Validating the CogSleep Screener in older adults at a memory and cognition clinic

Psychometric and adherence considerations for high-frequency, smartphone-based cognitive screening protocols in older adults

Primary care detection of cognitive impairment leveraging health and consumer technologies in underserved US communities: protocol for a pragmatic randomised controlled trial of the MyCog paradigm

Study protocol for a randomised controlled trial of CBT vs antipsychotics vs both in 14–18-year-olds: Managing Adolescent first episode Psychosis: a feasibility study (MAPS)

Validation of the 6-Item Cognitive Impairment Test and the 4AT test for combined delirium and dementia screening in older Emergency Department attendees

Cognitive function and treatment response trajectories in first-episode schizophrenia: evidence from a prospective cohort study

Multivariable prediction of functional outcome after first-episode psychosis: a crossover validation approach in EUFEST and PSYSCAN

The Development of a Brief but Comprehensive Therapeutic Assessment Protocol for the Screening and Support of Youth in the Community to Address the Youth Mental Health Crisis

An Effective Test (EOmciSS) for Screening Older Adults With Mild Cognitive Impairment in a Community Setting: Development and Validation Study

Web-based cognitive screening in bipolar disorder: validation of the Internet-based Cognitive Assessment Tool in remote administration settings

Cognitive performance at first episode of psychosis and the relationship with future treatment resistance: Evidence from an international prospective cohort study