Abstract:Abstract Measuring cognitive function is essential for characterizing brain health and tracking cognitive decline in Alzheimer’s Disease and other neurodegenerative conditions. Current tools to accurately evaluate cognitive impairment typically rely on a battery of questionnaires administered during clinical visits which is essential for the acquisition of repeated measurements in longitudinal studies. Previous studies have shown that the remote data collection of passively monitored daily interaction with personal digital devices can measure motor signs in the early stages of synucleinopathies, as well as facilitate longitudinal patient assessment in the real-world scenario with high patient compliance. This was achieved by the automatic discovery of patterns in the time series of keystroke dynamics, i.e. the time required to press and release keys, by machine learning algorithms. In this work, our hypothesis is that the typing patterns generated from user-device interaction may reflect relevant features of the effects of cognitive impairment caused by neurodegeneration. We use machine learning algorithms to estimate cognitive performance through the analysis of keystroke dynamic patterns that were extracted from mechanical and touchscreen keyboard use in a dataset of cognitively normal (n = 39, 51% male) and cognitively impaired subjects (n = 38, 60% male). These algorithms are trained and evaluated using a novel framework that integrates items from multiple neuropsychological and clinical scales into cognitive subdomains to generate a more holistic representation of multifaceted clinical signs. In our results, we see that these models based on typing input achieve moderate correlations with verbal memory, non-verbal memory and executive function subdomains [Spearman’s ρ between 0.54 (P < 0.001) and 0.42 (P < 0.001)] and a weak correlation with language/verbal skills [Spearman’s ρ 0.30 (P < 0.05)]. In addition, we observe a moderate correlation between our typing-based approach and the Total Montreal Cognitive Assessment score [Spearman’s ρ 0.48 (P < 0.001)]. Finally, we show that these machine learning models can perform better by using our subdomain framework that integrates the information from multiple neuropsychological scales as opposed to using the individual items that make up these scales. Our results support our hypothesis that typing patterns are able to reflect the effects of neurodegeneration in mild cognitive impairment and Alzheimer’s disease and that this new subdomain framework both helps the development of machine learning models and improves their interpretability.

Modeling and Predicting Mobile Phone Touchscreen Transcription Typing Using an Integrated Cognitive Architecture.

CRTypist: Simulating Touchscreen Typing Behavior Via Computational Rationality

Usability Evaluation of Smartphone Keyboard Design from an Approach of Structural Equation Model.

Typing Everywhere with an EMG Keyboard: A Novel Myo Armband-Based HCI Tool

Reactions, accuracy and response complexity of numerical typing on touch screens.

TOAST: Ten-Finger Eyes-Free Typing on Touchable Surfaces.

TapType: Ten-finger text entry on everyday surfaces via Bayesian inference

AirTyping

A novel framework to estimate cognitive impairment via finger interaction with digital devices

Hand Posture's Effect on Touch Screen Text Input Behaviors: A Touch Area Based Study

Modeling Human Transcription Typing with Queuing Network-Model Human Processor (QN-MHP)

From 2D to 3D: Facilitating Single-Finger Mid-Air Typing on Virtual Keyboards with Probabilistic Touch Modeling

A Monte Carlo Simulation Approach for Quantitatively Evaluating Keyboard Layouts for Gesture Input

Investigating Gesture Typing for Indirect Touch

Evaluation of Typing Efficiency Using Language Model for the Chinese Typewriter

Typing on Any Surface: A Deep Learning-based Method for Real-Time Keystroke Detection in Augmented Reality

Integrating Human Behavior Modeling and Data Mining Techniques to Predict Human Errors in Numerical Typing

Mathematically modelling the effects of pacing, finger strategies and urgency on numerical typing performance with queuing network model human processor.

RESEARCH ON USER FAULT MODEL OF INTELLIGENT STROKES INPUT SYSTEM IN MOBILE DEVICES

I-Keyboard: Fully Imaginary Keyboard on Touch Devices Empowered by Deep Neural Decoder

A Scenario Adaptive Model for Predicting Error Rates in Moving Target Selection on Smartphones