Abstract:With the development of network technology, the number of gambling websites has grown dramatically, causing a threat to social stability. There are many machine learning-based methods are proposed to identify gambling websites by analyzing the URL, the text, and the images of the websites. Nevertheless, most of the existing methods ignore one important piece of information, i.e., the text within the website images. Only the visual features of images are extracted for detection, while the semantic features of texts on the images are ignored. However, these texts have key information clearly pointing to gambling websites, which can help us identify such websites more accurately. Therefore, how to fuse image and text multimodal data is a challenge that should be met.Motivated by this, in this paper, we propose a hybrid multimodal data fusion-based method for identifying gambling websites by extracting and fusing visual and semantic features of the website screenshots. First, we fine tune the pretrained ResNet34 model to train an image classifier and to extract visual features of webpage screenshots. Second, we extract textual content from webpage screenshots through the optical character recognition (OCR) technique. We use pretrained Word2Vec word vectors as the initial embedding layer and use Bi-LSTM to train a text classifier and extract semantic features of textual content on the screenshots. Third, we use self-attention to fuse the visual and semantic features and train a multimodal classifier. The prediction results of image, text, and multimodal classifiers are fused by the late fusion method to obtain the final prediction result. To demonstrate the effectiveness of the proposed method, we conduct experiments on the webpage screenshot dataset we collected. The experimental results indicate that OCR text on the webpage screenshots has strong semantic features and the proposed hybrid multimodal data fusion based method can effectively improve the performance in identifying gambling websites, with accuracy, precision, recall, and F1-score all over 99%.

Customs classification for cross-border e-commerce based on text-image adaptive convolutional neural network

Fine Classification Method of Product Image Based on Multi-Level Convolutional Neural Networks

Research on Classification of Cross-Border E-Commerce Products Based on Image Recognition and Deep Learning

Harmonized system code prediction of import and export commodities based on Hybrid Convolutional Neural Network with Auxiliary Network

Multimodal Approach for Harmonized System Code Prediction

Cross-border e-commerce platform for commodity automatic pricing model based on deep learning

Research on Intelligent Customization of Cross-Border E-Commerce Based on Deep Learning

Cross-Border E-Commerce Platform Logistics and Supply Chain Network Optimization Based on Deep Learning

A Comprehensive Solution for Deep-Learning Based Cargo Inspection to Discriminate Goods in Containers

A transfer learning-based convolutional neural network and its novel application in ship spare-parts classification

Explainable Product Classification for Customs

Chinese Multicategory Sentiment of E-Commerce Analysis Based on Deep Learning

A Hybrid Multimodal Data Fusion-Based Method for Identifying Gambling Websites

Automatic Container Code Recognition Via Faster-Rcnn

Deep Learning Based Model for Prohibited Goods Detection

Fashion Analysis With A Subordinate Attribute Classification Network

Multi-angle perception and convolutional neural network for service quality evaluation of cross-border e-commerce logistics enterprise

Identifying Gambling Websites with Co-training

Hybrid recommendation algorithm of cross-border e-commerce items based on artificial intelligence and multiview collaborative fusion

A Multimodal Late Fusion Model for E-Commerce Product Classification

An Adaptive Deep Learning Framework for Shipping Container Code Localization and Recognition