Abstract:Mood disorders (MD) are among the most common mental health conditions worldwide, significantly contributing to mortality, morbidity, and disability rates. Individuals generate digital traces through their interactions with wearable and consumer-grade personal digital devices. These traces can be collected, processed, and analyzed, offering a unique opportunity to quantify and monitor individuals with mental disorders in their natural living environments. Various forms of digital traces from the use of smartphones and other personal digital devices hold the potential to reveal new behavioral markers associated with depressive symptoms in patients with mood disorders. We conducted an observational longitudinal study, MoMo-Mood Pilot, involving a cohort of patients with major depressive disorder (MDD) and a healthy control group without any diagnosed mental disorder to confirm the feasibility of our digital phenotyping study design. Upon confirming feasibility, we conducted a year-long, more extensive study, MoMo-Mood, with a similar design. This study comprised (1) three subcohorts of patients with a major depressive episode (MDE), including those with either MDD, bipolar disorder (BD), or concurrent borderline personality disorder (BPD), as well as (2) a healthy control group without any diagnosed mental disorder. We investigated whether differences in behavioral patterns, as quantified by passively collected digital trace data, could be observed at the group level, i.e., patients vs. healthy controls. We studied the volume and temporal patterns of smartphone screen and application usage, communication, sleep, mobility, and physical activity. Additionally, we examined which of the passively quantified behavioral features and sociodemographic factors are associated with the presence of depression in study participants at different points in time. MoMo-Mood Pilot and MoMo-Mood recruited 201 participants. The pilot study enrolled 14 patients with MDD and 23 healthy controls. The participants completed a 2-phase study: the first two weeks (the active phase) involved collecting data from bed sensors, actigraphy, and smartphone data passively collected through the Niima platform. Additionally, participants had to actively engage with the study by answering five sets of questions daily, prompted on their smartphones throughout the day. In the second phase (the passive phase), which lasted up to 1 year, only passive smartphone data and bi-weekly Patient Health Questionnaire (PHQ-9) assessments were collected. The MoMo-Mood study, which had a similar setting to the pilot study, enrolled 164 participants: 133 were patients diagnosed with current MDE through structured interviews (85 with MDD, 27 with BPD, and 21 with BD), and 31 were healthy controls. Survival analysis was performed to compare the adherence of healthy controls and the patient subcohorts. Statistical tests were performed to compare behavioral patterns, and a generalized linear mixed model was used to assess the association between different factors and the presence of depression. The study design was proven feasible upon completing the pilot study. Survival analysis using the log-rank test showed no statistically significant difference in participants adherence between subcohorts. The overall communication volume was similar in both groups, but the weekly temporal patterns revealed distinct preferences as the controls typically made or received calls in the afternoon. Regarding location patterns, significant differences emerged: weekday location variance showed lower values for patients, and normalized entropy of location was also lower among patients. However, no differences were observed in weekend location patterns. The temporal communication patterns for controls were found to be more diverse than that of patients (MWU test, p<0.001). In contrast, patients' smartphone usage temporal patterns were more varied than the controls. Investigating mobile-derived behaviors and their relationship with the concurrent presence of depression, we observed that the duration of outgoing calls over the past two weeks was negatively correlated with the presence of depression. Conversely, longer durations of incoming calls and a larger proportion of time spent at home were positively associated with the presence of depression. This work demonstrates the design of a longitudinal digital phenotyping study harnessing data from a cohort of patients with depression. It also shows the important features and data streams for future analyses of behavioral markers of mood disorders. However, among outpatients with mild to moderate depressive disorders, their group-level differences from healthy controls in any of the modalities alone remain overall modest. Therefore, future studies need to be able to combine data from multiple domains and modalities to detect more subtle differences, identify individualized signatures, and combine passive monitoring data with clinical ratings.

Using Multimodal Data Collection System as a Research Tool in the Major Depressive Disorder Analysis: a cross-sectional study protocol

Multi-Modal and Multi-Task Depression Detection with Sentiment Assistance

Multimodal Assessment of Schizophrenia and Depression Utilizing Video, Acoustic, Locomotor, Electroencephalographic, and Heart Rate Technology: Protocol for an Observational Study

Multimodal Mental Health Digital Biomarker Analysis from Remote Interviews using Facial, Vocal, Linguistic, and Cardiovascular Patterns

Real Time Multimodal Depression Analysis

Semi-Structural Interview-Based Chinese Multimodal Depression Corpus Towards Automatic Preliminary Screening of Depressive Disorders

End-to-end multimodal system for depression detection from online recordings

A Multimodal Approach for Detection and Assessment of Depression Using Text, Audio and Video

Multimodal digital assessment of depression with actigraphy and app in Hong Kong Chinese

Tracking and Monitoring Mood Stability of Patients with Major Depressive Disorder by Machine Learning Models Using Passive Digital Data: Prospective Naturalistic Multicenter Study

A multimodal dialog approach to mental state characterization in clinically depressed, anxious, and suicidal populations

Multimodal Sensing for Depression Risk Detection: Integrating Audio, Video, and Text Data

Multimodal Depression Severity Prediction from medical bio-markers using Machine Learning Tools and Technologies

Textual-dominated Multimodal Depression Detection

Using Wearable Devices and Speech Data for Personalized Machine Learning in Early Detection of Mental Disorders: Protocol for a Participatory Research Study

Classifying Major Depression Patients and Healthy Controls Using EEG, Eye Tracking and Galvanic Skin Response Data

MODMA dataset: a Multi-modal Open Dataset for Mental-disorder Analysis

Mobile Monitoring of Mood (MoMo-Mood): a Multimodal Digital Phenotyping Study with Major Depressive Patients and Healthy Controls

Adolescent Depression Detection Model Based on Multimodal Data of Interview Audio and Text.

Unaligned Multimodal Sequences for Depression Assessment From Speech

Depression Detection System