StopHC: A Harmful Content Detection and Mitigation Architecture for Social Media Platforms

Ciprian-Octavian Truică,Ana-Teodora Constantinescu,Elena-Simona Apostol

2024-11-09

Abstract:The mental health of social media users has started more and more to be put at risk by harmful, hateful, and offensive content. In this paper, we propose \textsc{StopHC}, a harmful content detection and mitigation architecture for social media platforms. Our aim with \textsc{StopHC} is to create more secure online environments. Our solution contains two modules, one that employs deep neural network architecture for harmful content detection, and one that uses a network immunization algorithm to block toxic nodes and stop the spread of harmful content. The efficacy of our solution is demonstrated by experiments conducted on two real-world datasets.

Social and Information Networks,Computation and Language

What problem does this paper attempt to address?

This paper attempts to solve the problem of detecting and mitigating harmful content on social media platforms. Specifically, the author proposes an architecture named S TOPHC, aiming to create a safer online environment, which is achieved through the following two main modules:

1. **Harmful Content Detection Module**: Utilize deep neural network models (such as BiLSTM, CNN, etc.) and different embedding techniques (such as Word2Vec, GloVe, BERT, RoBERTa and Node2Vec) to detect harmful content from text data. These models can capture the grammar, context and network information of the text, thus improving the detection accuracy.

2. **Network Immunity Module**: Use network immunity algorithms (such as Highest Degree, NetShield and DAVA) to prevent the spread of harmful content. This module identifies and blocks the key nodes that spread harmful content by analyzing the nodes and edges in the user interaction graph, thereby reducing the spread of harmful content.

The main contributions of the paper include:
- Proposing S TOPHC, a new architecture for harmful content detection and mitigation.
- Developing new deep neural network models for harmful content detection.
- Using immunity strategies to prevent the spread of harmful content on social media platforms.
- Conducting extensive experimental evaluations on two real - world datasets to verify the effectiveness of the scheme.

Through these methods, S TOPHC can not only accurately detect harmful content, but also effectively prevent its spread, thus protecting users' mental health and safety.