Abstract:The proliferation of social network data has unlocked unprecedented opportunities for extensive, data-driven exploration of human behavior. The structural intricacies of social networks offer insights into various computational social science issues, particularly concerning social influence and information diffusion. However, modeling large-scale social network data comes with computational challenges. Though large language models make it easier than ever to model textual content, any advanced network representation methods struggle with scalability and efficient deployment to out-of-sample users. In response, we introduce a novel approach tailored for modeling social network data in user detection tasks. This innovative method integrates localized social network interactions with the capabilities of large language models. Operating under the premise of social network homophily, which posits that socially connected users share similarities, our approach is designed to address these challenges. We conduct a thorough evaluation of our method across seven real-world social network datasets, spanning a diverse range of topics and detection tasks, showcasing its applicability to advance research in computational social science.
What problem does this paper attempt to address?
### Problems the Paper Attempts to Solve
This paper aims to address computational challenges in modeling large-scale social network data, particularly in user detection tasks. Specifically, although large language models (LLMs) have made it easier to model textual content, existing advanced network representation methods still face issues of scalability and efficient deployment when dealing with large-scale social network data. The paper proposes a new method—Social-LLM, which combines local social network interactions and the capabilities of large language models to address these issues.
### Main Contributions
1. **Proposing the Social-LLM Model**: This is a scalable social network representation model that combines user content cues and social network cues for inductive user detection tasks.
2. **Comprehensive Evaluation**: Social-LLM is thoroughly evaluated on 7 real-world social network datasets, covering various topics and detection tasks.
3. **Visualization Application**: Demonstrates the practicality of using Social-LLM embeddings for visualization.
### Method Overview
1. **Content Cues**: Mainly extracted from users' social media text content, including user profiles, metadata features (such as follower count, account creation date, etc.), and user tweets.
2. **Network Cues**: Formed through social interactions (such as retweets, mentions, follows) to create a social network. The paper uses the Twitter API to obtain three types of social interactions: retweets, mentions, and follows.
3. **Social-LLM Framework**:
- **User Representation Module**: Encodes user profiles using a pre-trained LLM model and connects the output into a single embedding vector through dense layers.
- **Unsupervised Training**: Optimizes user embeddings through a Siamese architecture, considering multiple edge types and directions.
- **Multi-Negative Ranking Loss**: Trains the model using a ranking loss function, comparing positive and negative examples.
- **Downstream Task Application**: The generated user representations can be applied to various downstream user prediction tasks, which can be further optimized through fine-tuning or freezing layers.
### Advantages and Disadvantages
**Advantages**:
- **Ease of Training**: Low time complexity, capable of handling very large datasets.
- **Inductive Capability**: Can provide embeddings for any new user without retraining the entire model.
- **Reusability**: The embedding training process is separated from downstream applications, allowing the learned embeddings to be reused.
**Disadvantages**:
- **Sacrificing Precision and Comprehensiveness**: Focuses only on first-order proximity, potentially losing valuable information from the global network structure or higher-order proximities.
### Datasets
The paper uses multiple datasets to validate the effectiveness of the method, including:
- **Covid-Political** and **Election2020**: For predicting political inclination.
- **COVID-Morality**: For predicting users' moral foundations.
- **Ukr-Rus-Suspended** and **Ukr-Rus-Hate**: For predicting account suspension and user toxicity levels, respectively.
- **Immigration-Hate-08** and **Immigration-Hate-05**: For predicting immigration hate speech.
### Evaluation
The paper evaluates Social-LLM in the second step of the overall workflow (Figure 1), extensively comparing it with a series of baseline methods and conducting sensitivity and ablation studies. Baseline methods are categorized into three types: content-based methods, network-based methods, and hybrid methods.