Abstract:The process of Topic Detection from Short Text Systems (SMS) is to extract distinct topics hidden inside short text collections, such as Twitter, Weibo, and instant messages. With the recent emergence of large volume user generated content collections enabled by online social media, topic detection from SMS becomes a challenging yet promising means for online public opinion analysis. In available literature, many forms and methods of topic detection have been proposed, but obtaining meaningful and coherent data is still difficult to reliably obtain for the extreme sparsity brought by SMS. To this end, we developed a Term-based Consensus Clustering topic detection (TCC) framework to provide an unsupervised methodology for finding distinct topics from within SMS collections. Specifically, we adopt a consensus clustering technique called K-means-based Consensus Clustering to handle SMS clustering, due to its low computational complexity and robust clustering performance. To further enrich the features of the information of the sparse SMS data, we conduct term clustering in the highly dense term space instead of the conventionally targeted sparse document space. To be more specific, we first use a feature space transfer technique to represent short text collections as a pseudo-document matrix, where rows, namely instances, correspond to terms and columns, namely features, correspond to adjacent terms. Basic partitions are generated from the pseudo-document matrix for term clustering and consensus clustering is followed to obtain the final term clustering result. Finally, a document classification process is adopted and a document is assigned to a cluster, where most terms occurred. Extensive experiments on real-world data sets demonstrate that TCC is comparable to several widely used methods in terms of topic detection quality. Particularly, we demonstrate that TCC obtains best clustering performance when observing a large number of the predefined topics across short text collections.

Text Clustering on Short Message by Using Deep Semantic Representation

Research on Neural Network Clustering Algorithm for Short Text

Short text clustering based on word embeddings and EMD

An Unsupervised Learning Short Text Clustering Method

A Short Text Clustering Approaches in Social Media

Clustering Text Data Streams

Clustering Massive Text Data Streams by Semantic Smoothing Model

Understanding Short Texts Through Semantic Enrichment and Hashing

Research on Short Text Representation Based on Sentential Semantic Components

Deep Structured Clustering of Short Text

Short Text Clustering Algorithms for Weibo Topic Detection

Transferring Topical Knowledge from Auxiliary Long Texts for Short Text Clustering

High-Efficiency Text Clustering Algorithm Based on Semantic Distance

Self-Taught convolutional neural networks for short text clustering

A Novel Text Clustering Algorithm Based on Inner Product Space Model of Semantic

Feature Dimension Reduction Short Text Clustering Combined with Semantic and Statistics

Topic Detection from Short Text: A Term-based Consensus Clustering Method

Research on K-means Text Clustering Algorithm Based on Semantic

Text Clustering Based on Feature Space

Text Clustering Via Term Semantic Units