Abstract:In the information explosion era, large scale data processing and mining is a hot issue. As microblog grows more popular, microblog services have become information provider on a web scale, so researches on microblog begin to focus more on its content mining than solely user's relationship analysis before. Although traditional text mining methods have been studied well, no algorithm is designed specially for microblog data, which contain structured information on social network besides plain text. In this paper, we introduce a novel probabilistic generative model MicroBlog-Latent Dirichlet Allocation (MB-LDA), which takes both contactor relevance relation and document relevance relation into consideration to improve topic mining in microblogs. Through Gibbs sampling for approximate inference of our model, MB-LDA can discover not only the topics of microblogs, but also the topics focused by contactors. When faced with large datasets, traditional techniques on single node become less practical within limited resources. So we present distributed MB-LDA in MapReduce framework in order to process large scale microblogs with high scalability. Furthermore, we apply a performance model to optimize the execution time by tuning the number of mappers and reducers. Experimental results on actual dataset show MB-LDA outperforms the baseline of LDA and distributed MB-LDA offers an effective solution to topic mining for large scale microblogs.

Infer the Probability of Read in Microblogs

Discerning Influence Patterns with Beta-Poisson Factorization in Microblogging Environments

Microblog Users' Life Time Activity Prediction

A Content-Based Intelligent Ranking Model for Microblog

Popularity prediction in microblog based on LR-DT

Research of Forwarding Behaviors of Users in Micro Blogs

A Cluster-Based Epidemic Model for Retweeting Trend Prediction on Micro-blog.

Information Propagation in Microblog Networks

Predicting lifespans of popular tweets in microblog.

ReTweeting Analysis and Prediction in Microblogs:An Epidemic Inspired Approach

Predicting Popularity of Topic Based on Similarity Relation and Co-occurrence Relation

Revealing the Efficiency of Information Diffusion in Online Social Networks of Microblog

Retweet Behavior Prediction Using Topic Model

A Real-Time Method to Predict Social Media Popularity

A Cascading Diffusion Prediction Model in Micro-blog Based on Multi-dimensional Features.

Large scale microblog mining using distributed MB-LDA.

User-level Microblogging Recommendation Incorporating Social Influence.

Characterizing Cascade Dynamics in a Microblogging System

A tweet-centric approach for topic-specific author ranking in micro-blog

Modeling collective behavior of posting microblogs by stochastic differential equation with jump

A Probabilistic Model For Bursty Topic Discovery In Microblogs