Abstract:With the explosion of online communication and publication, texts become obtainable via forums, chat messages, blogs, book reviews and movie reviews. Usually, these texts are much short and noisy without sufficient statistical signals and enough information for a good semantic analysis. Traditional natural language processing methods such as Bow-of-Word (BOW) based probabilistic latent semantic models fail to achieve high performance due to the short text environment. Recent researches have focused on the correlations between words, i.e., term dependencies, which could be helpful for mining latent semantics hidden in short texts and help people to understand them. Long short-term memory (LSTM) network can capture term dependencies and is able to remember the information for long periods of time. LSTM has been widely used and has obtained promising results in variants of problems of understanding latent semantics of texts. At the same time, by analyzing the texts, we find that a number of keywords contribute greatly to the semantics of the texts. In this paper, we establish a keyword vocabulary and propose an LSTM-based model that is sensitive to the words in the vocabulary; hence, the keywords leverage the semantics of the full document. The proposed model is evaluated in a short-text sentiment analysis task on two datasets: IMDB and SemEval-2016, respectively. Experimental results demonstrate that our model outperforms the baseline LSTM by 1%~2% in terms of accuracy and is effective with significant performance enhancement over several non-recurrent neural network latent semantic models (especially in dealing with short texts). We also incorporate the idea into a variant of LSTM named the gated recurrent unit (GRU) model and achieve good performance, which proves that our method is general enough to improve different deep learning models.

Utilizing Recurrent Neural Network for Topic Discovery in Short Text Scenarios

Don't Forget the Quantifiable Relationship between Words: Using Recurrent Neural Network for Short Text Topic Discovery.

Short Text Understanding by Leveraging Knowledge into Topic Model.

Topic Modeling for Short Texts Via Word Embedding and Document Correlation

A Rare Topic Discovery Model for Short Texts Based on Co-occurrence word Network

Robust Word-Network Topic Model For Short Texts

Topic Model over Short Texts Incorporating Word Embedding

Relational Biterm Topic Model: Short-Text Topic Modeling Using Word Embeddings

Context Reinforced Neural Topic Modeling over Short Texts

A Biterm-Based Dirichlet Process Topic Model for Short Texts

A Joint Model Of Extended Lda And Ibtm Over Streaming Chinese Short Texts

Emphasizing Essential Words for Sentiment Classification Based on Recurrent Neural Networks

BTM: Topic Modeling over Short Texts

Using Topic Modeling Methods for Short-Text Data: A Comparative Analysis

Constructing Pseudo Documents With Semantic Similarity For Short Text Topic Discovery

A Nested Chinese Restaurant Topic Model for Short Texts with Document Embeddings

Modeling over Short Texts

Topic Discovery for Short Texts Using Word Embeddings

Domain Dictionary-Based Topic Modeling For Social Text

A short text topic discovery method for social network

Topic Discovery from Heterogeneous Texts