Abstract:Background: Mental illness affects a significant portion of the worldwide population. Online mental health forums can provide a supportive environment for those afflicted and also generate a large amount of data that can be mined to predict mental health states using machine learning methods. Objective: This study aimed to benchmark multiple methods of text feature representation for social media posts and compare their downstream use with automated machine learning (AutoML) tools. We tested on datasets that contain posts labeled for perceived suicide risk or moderator attention in the context of self-harm. Specifically, we assessed the ability of the methods to prioritize posts that a moderator would identify for immediate response. Methods: We used 1588 labeled posts from the Computational Linguistics and Clinical Psychology (CLPsych) 2017 shared task collected from the Reachout.com forum. Posts were represented using lexicon-based tools, including Valence Aware Dictionary and sEntiment Reasoner, Empath, and Linguistic Inquiry and Word Count, and also using pretrained artificial neural network models, including DeepMoji, Universal Sentence Encoder, and Generative Pretrained Transformer-1 (GPT-1). We used Tree-based Optimization Tool and Auto-Sklearn as AutoML tools to generate classifiers to triage the posts. Results: The top-performing system used features derived from the GPT-1 model, which was fine-tuned on over 150,000 unlabeled posts from Reachout.com. Our top system had a macroaveraged F1 score of 0.572, providing a new state-of-the-art result on the CLPsych 2017 task. This was achieved without additional information from metadata or preceding posts. Error analyses revealed that this top system often misses expressions of hopelessness. In addition, we have presented visualizations that aid in the understanding of the learned classifiers. Conclusions: In this study, we found that transfer learning is an effective strategy for predicting risk with relatively little labeled data and noted that fine-tuning of pretrained language models provides further gains when large amounts of unlabeled text are available.

Online Social Network Post Classification: A Multiclass approach

Feature-Based Classification of Networks

A new ontology-based multimodal classification system for social media images of personality traits

Text Classification Using Hybrid Machine Learning Algorithms on Big Data

Exploring Hybrid and Ensemble Models for Multiclass Prediction of Mental Health Status on Social Media

Automating the Temperament Assessment of Online Social Network Users

Detecting Automatically Managed Accounts in Online Social Networks: Graph Embedding Approach

Transfer Learning for Risk Classification of Social Media Posts: Model Evaluation Study

Multi-level aspect based sentiment classification of Twitter data: using hybrid approach in deep learning

A K-means++ Based User Classification Method for Social E-commerce

An Ensemble-Based Multi-Classification Machine Learning Classifiers Approach to Detect Multiple Classes of Cyberbullying

Applying explainable artificial intelligence methods to models for diagnosing personal traits and cognitive abilities by social network data

Short text classification with machine learning in the social sciences: The case of climate change on Twitter

Classification of Instagram fake users using supervised machine learning algorithms

Structure Function Based Transform Features for Behavior-Oriented Social Media Image Classification

A Classification Framework for Online Social Support Using Deep Learning

Advancing Semantic Classification: A Comprehensive Examination of Machine Learning Techniques in Analyzing Russian-Language Patient Reviews

Dominant Set-based Active Learning for Text Classification and its Application to Online Social Media

Personality Classification of Social Users Based on Feature Fusion

Multimodal Classification for Analysing Social Media

A Hybrid Classification Algorithm to Classify Engineering Students' Problems and Perks