Abstract:Background Depression, anxiety, and apathy are highly prevalent in older people with preclinical dementia and mild cognitive impairment. These symptoms have also proven valuable in predicting the progression from mild cognitive impairment to dementia, enabling a timely diagnosis and treatment. However, objective and reliable indicators to detect and distinguish depression, anxiety, and apathy are relatively scarce. Objective This study aimed to develop a machine learning model to detect and distinguish depression, anxiety, and apathy based on speech and facial expressions. Design An observational, cross-sectional study design. Setting(s) The memory outpatient department of a tertiary hospital. Participants 319 older adults diagnosed with mild cognitive impairment. Methods Depression, anxiety, and apathy were evaluated by Public Health Questionnaire, General Anxiety Disorder, and Apathy Evaluation Scale, respectively. Speech and facial expressions of older adults with mild cognitive impairment were digitally captured using audio and video recording software. Open-source data analysis toolkits were utilized to extract speech, facial, and text features. The multiclass classification was used to develop classification models, and shapely additive explanations were used to explain the contribution of each feature within the model. Results The random forest method was used to develop a multiclass emotion classification model, which performed well in classifying emotions with a weighted-average F1 score of 96.6%. The model also demonstrated high accuracy, precision, and recall, with 87.4%, 86.6%, and 87.6%, respectively. Conclusions The machine learning model developed in this study demonstrated strong classification performance in detecting and differentiating depression, anxiety, and apathy. This innovative approach combines text, audio, and video to provide objective methods for precise classification and remote monitoring of these symptoms in nursing practice. Registration This study was registered at the Chinese Clinical Trial Registry (registration number: ChiCTR19000*****; registration date: June **th, 2019).

Cross-Lingual Cross-Age Group Adaptation for Low-Resource Elderly Speech Emotion Recognition

Real-Time Svm-Based Emotion Recognition Algorithm

Cross-Culture Multimodal Emotion Recognition With Adversarial Learning

Is Everything Fine, Grandma? Acoustic and Linguistic Modeling for Robust Elderly Speech Emotion Recognition

Real-time multimodal emotion recognition system based on elderly accompanying robot

Cross-Corpus Speech Emotion Recognition Based on Hybrid Neural Networks

Sentiment recognition of Italian elderly through domain adaptation on cross-corpus speech dataset

Compensating for age limits through emotional crossmodal integration

A cross-corpus study on speech emotion recognition

Embedded Emotions -- A Data Driven Approach to Learn Transferable Feature Representations from Raw Speech Input for Emotion Recognition

Learning Fine-Grained Cross Modality Excitement for Speech Emotion Recognition

Cross lingual speech emotion recognition via triple attentive asymmetric convolutional neural network

Linguistic-Coupled Age-to-Age Voice Translation to Improve Speech Recognition Performance in Real Environments

On-the-Fly Feature Based Rapid Speaker Adaptation for Dysarthric and Elderly Speech Recognition

Enhanced Facial Expression Recognition by Age

Speech Emotion Recognition in People at High Risk of Dementia

Exploiting Cross-domain And Cross-Lingual Ultrasound Tongue Imaging Features For Elderly And Dysarthric Speech Recognition

Unsupervised Cross-Lingual Speech Emotion Recognition Using DomainAdversarial Neural Network

Cross-lingual Speech Emotion Recognition: Humans vs. Self-Supervised Models

Design of Aging Smart Home Products Based on Radial Basis Function Speech Emotion Recognition

Developing a machine learning model for detecting depression, anxiety, and apathy in older adults with mild cognitive impairment using speech and facial expressions: A cross-sectional observational study