Abstract:Mobile sensing data processed using machine learning models can passively and remotely assess mental health symptoms from the context of patients' lives. Prior work has trained models using data from single longitudinal studies, collected from demographically homogeneous populations, over short time periods, using a single data collection platform or mobile application. The generalizability of model performance across studies has not been assessed. This study presents a first analysis to understand if models trained using combined longitudinal study data to predict mental health symptoms generalize across current publicly available data. We combined data from the CrossCheck (individuals living with schizophrenia) and StudentLife (university students) studies. In addition to assessing generalizability, we explored if personalizing models to align mobile sensing data, and oversampling less-represented severe symptoms, improved model performance. Leave-one-subject-out cross-validation (LOSO-CV) results were reported. Two symptoms (sleep quality and stress) had similar question-response structures across studies and were used as outcomes to explore cross-dataset prediction. Models trained with combined data were more likely to be predictive (significant improvement over predicting training data mean) than models trained with single-study data. Expected model performance improved if the distance between training and validation feature distributions decreased using combined versus single-study data. Personalization aligned each LOSO-CV participant with training data, but only improved predicting CrossCheck stress. Oversampling significantly improved severe symptom classification sensitivity and positive predictive value, but decreased model specificity. Taken together, these results show that machine learning models trained on combined longitudinal study data may generalize across heterogeneous datasets. We encourage researchers to disseminate collected de-identified mobile sensing and mental health symptom data, and further standardize data types collected across studies to enable better assessment of model generalizability.

Leveraging Large Language Models for Generating Mobile Sensing Strategies in Human Behavior Modeling

AWARE Narrator and the Utilization of Large Language Models to Extract Behavioral Insights from Smartphone Sensing Data

From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models

Characterizing a User from Large-Scale Smartphone-Sensed Data.

User Behavior Simulation with Large Language Model based Agents

Machine learning for passive mental health symptom prediction: Generalization across different longitudinal mobile sensing studies.

Context-Aware Behavioral Tips to Improve Sleep Quality via Machine Learning and Large Language Models

Inferring Mood Instability Via Smartphone Sensing

Leveraging LLMs to Predict Affective States via Smartphone Sensor Features

Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data

Human Mobility Modeling with Limited Information via Large Language Models

Where Would I Go Next? Large Language Models as Human Mobility Predictors

Leveraging Large Language Models for Mobile App Review Feature Extraction

From Twitter to Reasoner: Understand Mobility Travel Modes and Sentiment Using Large Language Models

Enabling On-Device LLMs Personalization with Smartphone Sensing

Semantic Enhancement of Human Urban Activity Chain Construction Using Mobile Phone Signaling Data

MindShift: Leveraging Large Language Models for Mental-States-Based Problematic Smartphone Use Intervention

A Spatial-Temporal Topic Segmentation Model For Human Mobile Behavior

Human Simulacra: Benchmarking the Personification of Large Language Models

Regions are Who Walk Them: a Large Pre-trained Spatiotemporal Model Based on Human Mobility for Ubiquitous Urban Sensing

Extending Coverage of Stationary Sensing Systems with Mobile Sensing Systems for Human Mobility Modeling