Detection of Illicit Drug Trafficking Events on Instagram: A Deep Multimodal Multilabel Learning Approach

Chuanbo Hu,Minglei Yin,Bin Liu,Xin Li,Yanfang Ye

DOI: https://doi.org/10.1145/3459637.3481908

2021-08-23

Abstract:Social media such as Instagram and Twitter have become important platforms for marketing and selling illicit drugs. Detection of online illicit drug trafficking has become critical to combat the online trade of illicit drugs. However, the legal status often varies spatially and temporally; even for the same drug, federal and state legislation can have different regulations about its legality. Meanwhile, more drug trafficking events are disguised as a novel form of advertising commenting leading to information heterogeneity. Accordingly, accurate detection of illicit drug trafficking events (IDTEs) from social media has become even more challenging. In this work, we conduct the first systematic study on fine-grained detection of IDTEs on Instagram. We propose to take a deep multimodal multilabel learning (DMML) approach to detect IDTEs and demonstrate its effectiveness on a newly constructed dataset called multimodal IDTE(MM-IDTE). Specifically, our model takes text and image data as the input and combines multimodal information to predict multiple labels of illicit drugs. Inspired by the success of BERT, we have developed a self-supervised multimodal bidirectional transformer by jointly fine-tuning pretrained text and image encoders. We have constructed a large-scale dataset MM-IDTE with manually annotated multiple drug labels to support fine-grained detection of illicit drugs. Extensive experimental results on the MM-IDTE dataset show that the proposed DMML methodology can accurately detect IDTEs even in the presence of special characters and style changes attempting to evade detection.

Machine Learning,Artificial Intelligence,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to accurately detect illegal drug - trading events (IDTEs) on Instagram. Specifically, the researchers face the following challenges:

1. **Inconsistency in the legal status of drugs**:
- The legal status of drugs varies by region and over time. For example, in the United States, marijuana is illegal under federal law, but in some states, its medical and recreational use is legal or decriminalized. Such changes in legal status make it difficult to distinguish between legal and illegal drug - trading.

2. **Information heterogeneity**:
- Information sources related to drug - trading include images and text, and the forms of advertising are diverse, ranging from direct posting to indirect comments (disguised as advertising). In addition, illegal drug - traders evade detection by changing font styles, adding separators, or creating new street names, increasing the complexity and diversity of the information.

3. **Accuracy**:
- The proportion of illegal drug - trading activities in social media data is relatively small, so finding these activities in a large amount of data is like looking for a needle in a haystack. How to efficiently mine data while maintaining a low false - positive rate is a technical challenge.

To solve these problems, the authors propose a method based on deep multimodal multi - label learning (DMML), aiming to detect the presence and types of multiple illegal drugs from user activities (such as posts and comments) on Instagram. The specific methods are as follows:

- **Multimodal information fusion**: Combine text and image data, and perform feature extraction and fusion through the self - supervised bidirectional Transformer (MMBT) model.
- **Multi - label learning module**: Predict multiple illegal drug labels to achieve fine - grained illegal drug - trading detection.

In addition, the authors also construct a large - scale multimodal illegal drug - trading event (MM - IDTE) dataset to support research on fine - grained illegal drug detection. Experimental results show that this method has high accuracy and robustness in detecting illegal drug - trading events.