Early Prediction of Full Mastery of 6 Technology-Based Functional Skills in a Computerized Skills Training Program for Individuals with Mild Cognitive Impairment
Courtney Dowell-Esquivel,Philip D. Harvey,Sara J. Czaja,Peter Kallestrup,Justin Macchiarelli,Alajandro Martinez
DOI: https://doi.org/10.1016/j.jagp.2024.01.164
2024-04-01
Abstract:Introduction Cognitive impairments in individuals with mild cognitive impairment (MCI) have responded poorly to pharmacological interventions, leading to the use of various computerized training strategies. In addition to previously reported improvement in cognitive performance with computerized cognitive training (CCT), a combination of CCT and computerized functional skills training software (FUNSAT) produced improvements in the performance of 6 different functional skills in participants with MCI, with effects sizes for gains close to 1.0 SD. While full mastery of all 6 skills was common in participants with normal cognition (NC), about 50% of MCI participants failed to master all FUNSAT tasks. Early identification of eventual failure to master the tasks would provide guidance for alternative training strategies. In an updated study, fully remote training was used and criteria for mastery were adjusted to be easier for participants to understand. This presentation addresses the identification of early performance characteristics, either baseline performance or features of early training gains, to attempt to identify participants who do not achieve full mastery with a goal of modifying either training procedures or instructions in later versions of the software. Methods NC participants (n=71) received skills training alone and MCI (n=83) participants received skills training alone or combined CCT and skills. Participants trained 2 times/week for up to 12 weeks on a fully remote basis. Tasks were introduced in a fixed order. Training gains were indexed by two outcomes: completion time and errors. There were six different functional skills, each with 3 to 6 subtasks, that were trained by the program and participants “graduated” each subtask when they made one or fewer errors in each of the subtasks. Subtasks were no longer trained after mastery and each overall task was designated as “graduated” after all subtasks were mastered. In this presentation, we limited our analyses to the three easiest tasks: Ticket Purchase, ATM Banking, and Medication Management, because they were also the first three tasks presented to participants. Results 83 MCI participants completed training, of whom 24 (29%) participants failed to master all tasks. For the NC participants, mastery was achieved by 69 (97%) of the participants. Between groups comparisons with the MCI populations were conducted on completion time and errors across tasks at baseline. When these 6 variables were compared with t-tests across mastery, completion time did not predict eventual mastery for any tasks, while baseline errors separated by an effect size of greater than 0.5 SD across all tasks predicted mastery and was statistically significant. When NC individuals were also included and errors across the three tasks were used to predict mastery of all tasks in a discriminant analysis, it was found that ticket task errors uniquely separated the groups, F=41.40, p<.001. When the analysis was repeated and training gains on the first session for all three tasks were also added to the discriminant analysis, training gains on the first session for the ticket task entered the analysis after baseline errors on the ticket test, but to a modestly lesser extent than baseline errors: F=39.66, p<.001. An ROC analysis found an AUC for ticket errors at baseline of .83, with correct identification of 94% of the eventual graduates with ticket task baseline errors as a sole predictor. Conclusions More baseline errors, but not completion times, predicted failure to eventually master functional skills training tasks within all participants with MCI. Importantly, the most predictive task was also objectively the easiest and was the first task administered. When all participants were examined with multivariate predictive analyses, accuracy of identification of eventual mastery was exceptional. Thus, it is feasible to identify the risk of failure to attain task mastery through early detection, often identifiable within the first 15 minutes of the first assessment. This information may be used to improve training strategies and facilitate mastery in impaired populations.
psychiatry,geriatrics & gerontology,gerontology