Abstract:Background: Monitoring the psychological conditions of social media users during rapidly developing public health crises, such as the COVID-19 pandemic, using their posts on social media has rapidly gained popularity as a relatively easy and cost-effective method. However, the characteristics of individuals who created these posts are largely unknown, making it difficult to identify groups of individuals most affected by such crises. In addition, large annotated data sets for mental health conditions are not easily available, and thus, supervised machine learning algorithms can be infeasible or too costly. Objective: This study proposes a machine learning framework for the real-time surveillance of mental health conditions that does not require extensive training data. Using survey-linked tweets, we tracked the level of emotional distress during the COVID-19 pandemic by the attributes and psychological conditions of social media users in Japan. Methods: We conducted online surveys of adults residing in Japan in May 2022 and collected their basic demographic information, socioeconomic status, and mental health conditions, along with their Twitter handles (N=2432). We computed emotional distress scores for all the tweets posted by the study participants between January 1, 2019, and May 30, 2022 (N=2,493,682) using a semisupervised algorithm called latent semantic scaling (LSS), with higher values indicating higher levels of emotional distress. After excluding users by age and other criteria, we examined 495,021 (19.85%) tweets generated by 560 (23.03%) individuals (age 18-49 years) in 2019 and 2020. We estimated fixed-effect regression models to examine their emotional distress levels in 2020 relative to the corresponding weeks in 2019 by the mental health conditions and characteristics of social media users. Results: The estimated level of emotional distress of our study participants increased in the week when school closure started (March 2020), and it peaked at the beginning of the state of emergency (estimated coefficient=0.219, 95% CI 0.162-0.276) in early April 2020. Their level of emotional distress was unrelated to the number of COVID-19 cases. We found that the government-induced restrictions disproportionately affected the psychological conditions of vulnerable individuals, including those with low income, precarious employment, depressive symptoms, and suicidal ideation. Conclusions: This study establishes a framework to implement near-real-time monitoring of the emotional distress level of social media users, highlighting a great potential to continuously monitor their well-being using survey-linked social media posts as a complement to administrative and large-scale survey data. Given its flexibility and adaptability, the proposed framework is easily extendable for other purposes, such as detecting suicidality among social media users, and can be used on streaming data for continuous measurement of the conditions and sentiment of any group of interest.

Urban-regional Disparities in Mental Health Signals in Australia During the COVID-19 Pandemic: a Study Via Twitter Data and Machine Learning Models

The times, they are a-changin’: tracking shifts in mental health signals from early phase to later phase of the COVID-19 pandemic in Australia

Discovery of the Environmental Factors Affecting Urban Dwellers’ Mental Health: A Data-Driven Approach

Tracking the impact of COVID-19 and lockdown policy on public mental health using social media

A global portrait of expressed mental health signals towards COVID-19 in social media space

Exploring the Effect of Social Media and Spatial Characteristics During the COVID-19 Pandemic in China

Detecting Community Depression Dynamics Due to COVID-19 Pandemic in Australia

Social media insights into spatio-temporal emotional responses to COVID-19 crisis

Examination of Community Sentiment Dynamics due to COVID-19 Pandemic: A Case Study from A State in Australia

Tracking the Impact of COVID-19 and Lockdown Policies on Public Mental Health Using Social Media: Infoveillance Study

Mental Health Pandemic During the COVID-19 Outbreak: Social Media As a Window to Public Mental Health

Analyzing the Relationship Between COVID-19 and Sociodemographic and Environmental Factors: A Case Study in Toronto

Mental health inequality between urban and rural youth under COVID-19 from survey data from China

A machine learning analysis of COVID-19 mental health data

Using Twitter Data to Estimate the Prevalence of Symptoms of Mental Disorders in the United States During the COVID-19 Pandemic: Ecological Cohort Study

Unpacking the inter- and intra-urban differences of the association between health and exposure to heat and air quality in Australia using global and local machine learning models

Social Behavior and Mental Health: A Snapshot Survey under COVID-19 Pandemic

Investigating the Civic Emotion Dynamics during the COVID-19 Lockdown: Evidence from Social Media

Emotional Distress During COVID-19 by Mental Health Conditions and Economic Vulnerability: Retrospective Analysis of Survey-Linked Twitter Data With a Semisupervised Machine Learning Algorithm

Exploring Social Media for Early Detection of Depression in COVID-19 Patients

Using Social Media Data to Assess the Impact of COVID-19 on Mental Health in China