Abstract:With the development of network technology, the number of gambling websites has grown dramatically, causing a threat to social stability. There are many machine learning-based methods are proposed to identify gambling websites by analyzing the URL, the text, and the images of the websites. Nevertheless, most of the existing methods ignore one important piece of information, i.e., the text within the website images. Only the visual features of images are extracted for detection, while the semantic features of texts on the images are ignored. However, these texts have key information clearly pointing to gambling websites, which can help us identify such websites more accurately. Therefore, how to fuse image and text multimodal data is a challenge that should be met.Motivated by this, in this paper, we propose a hybrid multimodal data fusion-based method for identifying gambling websites by extracting and fusing visual and semantic features of the website screenshots. First, we fine tune the pretrained ResNet34 model to train an image classifier and to extract visual features of webpage screenshots. Second, we extract textual content from webpage screenshots through the optical character recognition (OCR) technique. We use pretrained Word2Vec word vectors as the initial embedding layer and use Bi-LSTM to train a text classifier and extract semantic features of textual content on the screenshots. Third, we use self-attention to fuse the visual and semantic features and train a multimodal classifier. The prediction results of image, text, and multimodal classifiers are fused by the late fusion method to obtain the final prediction result. To demonstrate the effectiveness of the proposed method, we conduct experiments on the webpage screenshot dataset we collected. The experimental results indicate that OCR text on the webpage screenshots has strong semantic features and the proposed hybrid multimodal data fusion based method can effectively improve the performance in identifying gambling websites, with accuracy, precision, recall, and F1-score all over 99%.

Adult image recognition algorithm based on Bag-of-Visual-Words and text analysis

An Effective Method for Distinguishing Photograph and Graphics

Adult Image Detection Combining BoVW Based on Region of Interest and Color Moments.

Object Recognition Based on the Region of Interest and Optimal Bag of Words Model.

A Pca Based Automatic Image Categorization Approach Using Dominant Color Features

Sensitive Information Detection Based on Deep Learning Models

IMAGE GUARDER: AN INTELLIGENT DETECTOR FOR ADULT IMAGES

Adult Image Detection Method Base-On Skin Color Model And Support Vector Machine 1

Smart Content Recognition from Images Using a Mixture of Convolutional Neural Networks

Adult Image Detection Method Base-on Skin Color Model and Support

Combining SIFT and global features for web image classification

Detecting Web Pornographic Images Based on Local SIFT Features

The image annotation algorithm using convolutional features from intermediate layer of deep learning

Adaptive Model for Integrating Different Types of Associated Texts for Automated Annotation of Web Images

Automatic image annotation based on salient regions

Pornographic Image Detection Based on Multilevel Representation

Shape-Based Adult Image Detection

An erotic image recognition algorithm based on trunk model and SVM classification

Research on Web Pornographic Image Filtering

A Hybrid Multimodal Data Fusion-Based Method for Identifying Gambling Websites

Fusing Audio-Words with Visual Features for Pornographic Video Detection