Usability of the Mayo Test Drive remote self-administered web-based cognitive screening battery in adults ages 35 to 100 with and without cognitive impairment
Jay S Patel,Teresa J Christianson,Logan T Monahan,Ryan D Frank,Winnie Z Fan,John L Stricker,Walter K Kremers,Aimee J Karstens,Mary M Machulda,Julie A Fields,Jason Hassenstab,Clifford R. Jack Jr.,Hugo Botha,Jonathan Graff-Radford,Ronald C Petersen,Nikki Horne Stricker
DOI: https://doi.org/10.1101/2024.05.24.24307909
2024-05-28
Abstract:Background. Self-administered cognitive screening measures that can be completed unsupervised and remotely have significant promise for high scalability. Mayo Test Drive (MTD): Test Development through Rapid Iteration, Validation and Expansion, is a web-based remote cognitive assessment platform for self-administered neuropsychological measures with previously demonstrated validity and reliability. Our primary study aim was to examine the usability of MTD. We additionally described participation and adherence rates; characterized factors that inform feasibility to implement MTD in research and clinical settings such as session and subtest durations, frequency of interruptions and noise during test sessions, device type use across demographic and clinical characteristics for completed sessions; and reported qualitative themes of free-text participant comments.
Methods. 1,950 Mayo Clinic Study of Aging and Mayo Alzheimer Disease Research Center participants (97% White, 99% Non-Hispanic) were invited to participate in this ancillary, uncompensated remote study. Most invitees were cognitively unimpaired (CU; n=1,769; 90.7%) and 9.3% were cognitively impaired (CI; n=181). Participation and adherence rates were defined as the number of participants initiating a test session when invited or due to complete a session, respectively. Usability was objectively defined as the percentage of participants who completed a session after initiating a session for a given timepoint (i.e., completion rates). MTD sessions included the Stricker Learning Span (SLS) learning and delay trials, four trials of the Symbols test, and completion of self-report questions that informed session context. All participants were invited to provide free-text comments that were coded by investigators and descriptively characterized by qualitative analysis.
Results. Baseline session completion rates were 98.5% across participants who initiated a test session (n=1199/1217, mean age 71, SD=12, range 35-100) and were comparable between CU (98.7%) and CI (95.0%) groups (p=.23). Completion rates did not significantly differ by age groups among CU participants (p>.10) and completion rates remained high in individuals 80+ (n=251, 97.3%). Participation rates were higher in the CU (n=1142, 65.4%) versus CI (n=57, 33.1%) group (p<.001). Participation rates were lower in those 80+ (53.8%) relative to younger age groups (68.9% ages 34-64; 70.2% ages 65-69). Adherence (i.e., retention) rate for a 7.5-month follow-up session was 89%. Average session duration was 16 minutes. Most participants used a personal computer (n=751, 62.7%), followed by a smartphone (n=266, 22.2%) or tablet (n=177, 14.8%). Comments were entered by 36.4% of participants and reflected several themes relevant to acceptability, face validity, usability, as well as comments informative for session context.
Conclusions. MTD demonstrated high usability as defined by completion rates in this research sample that includes a broad age range, though participation rates are lower in individuals with cognitive impairment. Results support good adherence at follow-up, feasibility through mean session durations, and acceptability based on qualitative analysis of participant comments.