Abstract:Background: Advances in machine learning (ML) technology have opened new avenues for detection and monitoring of cognitive decline. In this study, a multimodal approach to Alzheimer's dementia detection based on the patient's spontaneous speech is presented. This approach was tested on a standard, publicly available Alzheimer's speech dataset for comparability. The data comprise voice samples from 156 participants (1:1 ratio of Alzheimer's to control), matched by age and gender. Materials and Methods: A recently developed Active Data Representation (ADR) technique for voice processing was employed as a framework for fusion of acoustic and textual features at sentence and word level. Temporal aspects of textual features were investigated in conjunction with acoustic features in order to shed light on the temporal interplay between paralinguistic (acoustic) and linguistic (textual) aspects of Alzheimer's speech. Combinations between several configurations of ADR features and more traditional bag-of-n-grams approaches were used in an ensemble of classifiers built and evaluated on a standardised dataset containing recorded speech of scene descriptions and textual transcripts. Results: Employing only semantic bag-of-n-grams features, an accuracy of 89.58% was achieved in distinguishing between Alzheimer's patients and healthy controls. Adding temporal and structural information by combining bag-of-n-grams features with ADR audio/textual features, the accuracy could be improved to 91.67% on the test set. An accuracy of 93.75% was achieved through late fusion of the three best feature configurations, which corresponds to a 4.7% improvement over the best result reported in the literature for this dataset. Conclusion: The proposed combination of ADR audio and textual features is capable of successfully modelling temporal aspects of the data. The machine learning approach toward dementia detection achieves best performance when ADR features are combined with strong semantic bag-of-n-grams features. This combination leads to state-of-the-art performance on the AD classification task.

Automatic Identification of Alzheimer's Disease using Lexical Features extracted from Language Samples

Identification of Alzheimer's Disease Patients Based on Oral Speech Features

Combining Prosodic, Voice Quality and Lexical Features to Automatically Detect Alzheimer's Disease

Automated free speech analysis reveals distinct markers of Alzheimer's and frontotemporal dementia

Temporal Integration of Text Transcripts and Acoustic Features for Alzheimer's Diagnosis Based on Spontaneous Speech

Using the Outputs of Different Automatic Speech Recognition Paradigms for Acoustic- and BERT-based Alzheimer's Dementia Detection through Spontaneous Speech

Classifying Alzheimer's Disease Using Audio and Text-Based Representations of Speech

Explainable Alzheimer's Disease Detection Using Linguistic Features from Automatic Speech Recognition

Detecting Linguistic Characteristics of Alzheimer's Dementia by Interpreting Neural Models

Influence of ASR and Language Model on Alzheimer's Disease Detection

Linguistic Features Extracted by GPT-4 Improve Alzheimer's Disease Detection based on Spontaneous Speech

Improving Detection of Alzheimer's Disease Using Automatic Speech Recognition to Identify High-Quality Segments for More Robust Feature Extraction.

Detecting Alzheimer's Disease Using Natural Language Processing of Referential Communication Task Transcripts

An approach for assisting diagnosis of Alzheimer's disease based on natural language processing

Detecting Alzheimer's Disease from Continuous Speech Using Language Models.

A Comparison of Connected Speech Tasks for Detecting Early Alzheimer’s Disease and Mild Cognitive Impairment Using Natural Language Processing and Machine Learning

Leveraging Pretrained Representations with Task-related Keywords for Alzheimer's Disease Detection

ML-Based Analysis to Identify Speech Features Relevant in Predicting Alzheimer's Disease

Analysis of Speech Features in Alzheimer's Disease with Machine Learning: A Case-Control Study

Comparing Natural Language Processing Techniques for Alzheimer's Dementia Prediction in Spontaneous Speech

Alzheimer's Dementia Detection from Audio and Text Modalities