Abstract:Background: Previous research has shown the feasibility of using machine learning models trained on social media data from a single platform (eg, Facebook or Twitter) to distinguish individuals either with a diagnosis of mental illness or experiencing an adverse outcome from healthy controls. However, the performance of such models on data from novel social media platforms unseen in the training data (eg, Instagram and TikTok) has not been investigated in previous literature. Objective: Our study examined the feasibility of building machine learning classifiers that can effectively predict an upcoming psychiatric hospitalization given social media data from platforms unseen in the classifiers' training data despite the preliminary evidence on identity fragmentation on the investigated social media platforms. Methods: Windowed timeline data of patients with a diagnosis of schizophrenia spectrum disorder before a known hospitalization event and healthy controls were gathered from 3 platforms: Facebook (254/268, 94.8% of participants), Twitter (51/268, 19% of participants), and Instagram (134/268, 50% of participants). We then used a 3 × 3 combinatorial binary classification design to train machine learning classifiers and evaluate their performance on testing data from all available platforms. We further compared results from models in intraplatform experiments (ie, training and testing data belonging to the same platform) to those from models in interplatform experiments (ie, training and testing data belonging to different platforms). Finally, we used Shapley Additive Explanation values to extract the top predictive features to explain and compare the underlying constructs that predict hospitalization on each platform. Results: We found that models in intraplatform experiments on average achieved an F 1 -score of 0.72 (SD 0.07) in predicting a psychiatric hospitalization because of schizophrenia spectrum disorder, which is 68% higher than the average of models in interplatform experiments at an F 1 -score of 0.428 (SD 0.11). When investigating the key drivers for divergence in construct validities between models, an analysis of top features for the intraplatform models showed both low predictive feature overlap between the platforms and low pairwise rank correlation (<0.1) between the platforms' top feature rankings. Furthermore, low average cosine similarity of data between platforms within participants in comparison with the same measurement on data within platforms between participants points to evidence of identity fragmentation of participants between platforms. Conclusions: We demonstrated that models built on one platform's data to predict critical mental health treatment outcomes such as hospitalization do not generalize to another platform. In our case, this is because different social media platforms consistently reflect different segments of participants' identities. With the changing ecosystem of social media use among different demographic groups and as web-based identities continue to become fragmented across platforms, further research on holistic approaches to harnessing these diverse data sources is required.

Detecting and Characterizing Eating-Disorder Communities on Social Media

Social interactions in online eating disorder communities: A network perspective

Characterizing dynamic communication in online eating disorder communities: a multiplex network approach

Analysis of Twitter to Identify Topics Related to Eating Disorder Symptoms.

What Topics Do Members of the Eating Disorder Online Community Discuss and Empathize with? An Application of Big Data Analytics

Exploring Eating Disorder Topics on Twitter: Machine Learning Approach.

Detecting Signals of Associations Between Dietary Supplement Use and Mental Disorders from Twitter

Detecting associations between dietary supplement intake and sentiments within mental disorder tweets.

Approaching what and how people with mental disorders communicate in social media–Introducing a multi-channel representation

Large Language Models Help Reveal Unhealthy Diet and Body Concerns in Online Eating Disorders Communities

A Novel Site-Agnostic Multimodal Deep Learning Model to Identify Pro-Eating Disorder Content on Social Media

Mining Online Social Data for Detecting Social Network Mental Disorders

RevealED: Uncovering Pro-Eating Disorder Content on Twitter Using Deep Learning

Mental Disorders on Online Social Media Through the Lens of Language and Behaviour: Analysis and Visualisation

Mining Twitter to Assess the Determinants of Health Behavior towards Palliative Care in the United States.

Social Behavior and Mental Health: A Snapshot Survey under COVID-19 Pandemic

Empowering machine learning models with contextual knowledge for enhancing the detection of eating disorders in social media posts

Emotion-based Modeling of Mental Disorders on Social Media

Characterization of Anorexia Nervosa on Social Media: Textual, Visual, Relational, Behavioral, and Demographical Analysis

Cross-Platform Detection of Psychiatric Hospitalization via Social Media Data: Comparison Study

Downloading appetite? Investigating the role of parasocial relationship with favorite social media food influencer in followers' disordered eating behaviors