Abstract:Background: An estimated 6.7 million persons are living with dementia in the United States, a number expected to double by 2060. Persons experiencing moderate to severe dementia are 4 to 5 times more likely to fall than those without dementia, due to agitation and unsteady gait. Socially assistive robots fail to address the changing emotional states associated with agitation, and it is unclear how emotional states change, how they impact agitation and gait over time, and how social robots can best respond by showing empathy. Objective: This study aims to design and validate a foundational model of emotional intelligence for empathetic patient-robot interaction that mitigates agitation among those at the highest risk: persons experiencing moderate to severe dementia. Methods: A design science approach will be adopted to (1) collect and store granular, personal, and chronological data using Personicle (an open-source software platform developed to automatically collect data from phones and other devices), incorporating real-time visual, audio, and physiological sensing technologies in a simulation laboratory and at board and care facilities; (2) develop statistical models to understand and forecast the emotional state, agitation level, and gait pattern of persons experiencing moderate to severe dementia in real time using machine learning and artificial intelligence and Personicle; (3) design and test an empathy-focused conversation model, focused on storytelling; and (4) test and evaluate this model for a care companion robot (CCR) in the community. Results: The study was funded in October 2023. For aim 1, architecture development for Personicle data collection began with a search for existing open-source data in January 2024. A community advisory board was formed and met in December 2023 to provide feedback on the use of CCRs and provide personal stories. Full institutional review board approval was received in March 2024 to place cameras and CCRs at the sites. In March 2024, atomic marker development was begun. For aim 2, after a review of open-source data on patients with dementia, the development of an emotional classifier was begun. Data labeling was started in April 2024 and completed in June 2024 with ongoing validation. Moreover, the team established a baseline multimodal model trained and validated on healthy-person data sets, using transformer architecture in a semisupervised manner, and later retrained on the labeled data set of patients experiencing moderate to severe dementia. In April 2024, empathy alignment of large language models was initiated using prompt engineering and reinforcement learning. Conclusions: This innovative caregiving approach is designed to recognize the signs of agitation and, upon recognition, intervene with empathetic verbal communication. This proposal has the potential to have a significant impact on an emerging field of computational dementia science by reducing unnecessary agitation and falls of persons experiencing moderate to severe dementia, while reducing caregiver burden. International registered report identifier (irrid): PRR1-10.2196/55761.

Sensor-based agitation prediction in institutionalized people with dementia A systematic review

A Novel Multimodal System to Predict Agitation in People with Dementia Within Clinical Settings: A Proof of Concept

Agitation Detection in People Living with Dementia using Multimodal Sensors

An attention model to analyse the risk of agitation and urinary tract infections in people with dementia

Toward Quantification of Agitation in People With Dementia Using Multimodal Sensing

Physiological Profiling of Agitation in Dementia: Insights From Wearable Sensor Data

Identifying pre-agitation biometric signature in patients with dementia: A feasibility study

P171: Identifying pre-agitation biometric signature in dementia patients: A feasibility study

Designing A Clinically Applicable Deep Recurrent Model to Identify Neuropsychiatric Symptoms in People Living with Dementia Using In-Home Monitoring Data

Detecting agitation and aggression in persons living with dementia: a systematic review of diagnostic accuracy

Undersampling and Cumulative Class Re-decision Methods to Improve Detection of Agitation in People with Dementia

An Interpretable Machine Learning Tool for In-Home Screening of Agitation Episodes in People Living with Dementia

Prediction of Dementia-related Agitation Using Multivariate Ambient Environmental Time-series Data

Sensortechnology for monitoring challenging behavior in nursing home residents with dementia

Using AI-Based Technologies to Help Nurses Detect Behavioral Disorders: Narrative Literature Review

Smart Home Sensing and Monitoring in Households With Dementia: User-Centered Design Approach

S14: The use of advanced data and sensortechnology in dementia: innovation and implementation

Semi-supervised Learning for Identifying the Likelihood of Agitation in People with Dementia

Ambient Activity Technologies for Managing Responsive Behaviours in Dementia

Non-pharmacological interventions for agitation in dementia: systematic review of randomised controlled trials

Establishing the Foundations of Emotional Intelligence in Care Companion Robots to Mitigate Agitation Among High-Risk Patients With Dementia: Protocol for an Empathetic Patient-Robot Interaction Study