Abstract:Short message services (SMS), microblogging tools, instant message apps, and commercial websites produce numerous short text messages every day. These short text messages are usually guaranteed to reach mass audience with low cost. Spammers take advantage of short texts by sending bulk malicious or unwanted messages. Short texts are difficult to classify because of their shortness, sparsity, rapidness, and informal writing. The effectiveness of the hidden Markov model (HMM) for short text classification has been illustrated in our previous study. However, the HMM has limited capability to handle new words, which are mostly generated by informal writing. In this paper, a hybrid model is proposed to address the informal writing issue by weighting new words for fast short text filtering with high accuracy. The hybrid model consists of an artificial neural network (ANN) and an HMM, which are used for new word weighting and spam filtering, respectively. The weight of a new word is calculated based on the weights of its neighbor, along with the spam and ham (i.e., not spam) probabilities of short text message predicted by the ANN. Performance evaluations on benchmark datasets, including the SMS message data maintained by University of California, Irvine; the movie reviews, and the customer reviews are conducted. The hybrid model operates at a significantly higher speed than deep learning models. The experiment results show that the proposed hybrid model outperforms other prominent machine learning algorithms, achieving a good balance between filtering throughput and accuracy.

Character-Based Language Modeling Approach for Spam Filtering

Implementation and Evaluation of Chinese Spam Filtering System

Effective spam filter based on a hybrid method of header checking and content parsing

Complex Network Based SMS Filtering Algorithm

Spam Message Self-Adaptive Filtering System Based on Naive Bayes and Support Vector Machine

TopicSpam: a Topic-Model Based Approach for Spam Detection.

Toward a Language Modeling Approach for Consumer Review Spam Detection

Largemargin Classification for Combating Disguise Attacks on Spam Filters

A Hybrid Model with New Word Weighting for Fast Filtering Spam Short Texts

Filtering Chinese Spam Email Using Logistic Regression

A New Spam Detection and Filtration Method

A Local-Concentration-Based Feature Extraction Approach for Spam Filtering.

A Content-Based Chinese Spam Detection Method Using a Capsule Network With Long-Short Attention

A Composite Intelligent Method For Spam Filtering

Novel method for Chinese spam detection based on one-class support vector machines

A Spam Filtering Method Based on Bayesian Neural Network

Content-based Spam Email Detection Using N-gram Machine Learning Approach

Classify E-mails by Support Vector Machine

Spam message online filtering system based on hash function and naive Bayes

Semorph: A Morphology Semantic Enhanced Pre-trained Model for Chinese Spam Text Detection.

Application of Natural Language Processing and Machine Learning Boosted with Swarm Intelligence for Spam Email Filtering