Abstract:A daily summary or digest from microblogs allows social media users to stay up to date on what happened today on their favorite topic. Summarizing microblogs is a non-trivial task. This paper presents a summarization system built over the Twitter stream to summarize the topic for a given duration. Tweet ranking is the primary task of designing a microblog-based summarization system. After ranking tweets, the selection of relevant tweets is the crucial task for any summarization system due to the massive volume of tweets in the Twitter stream. In addition, the summarization system should include novel tweets in the summary or digest. The measure of relevance is typically the similarity score obtained from different text similarity algorithms. These measure the similarity between user information needs and each tweet. The more similar, the higher the score. So we need to choose a threshold that can minimize false-positive judgments for this task. In this paper, we proposed novel threshold estimation methods to find optimal values for these thresholds and evaluate them against thresholds determined via grid search. These methods estimate the thresholds with reasonable accuracy, according to the results. Previous research has empirically and heuristically set these thresholds, and our work suggests a method that exploits statistical features of the ranking list to estimate these thresholds. We used language models to rank the tweets and to select relevant tweets. For any language model, the selection of the smoothing technique and its parameters are critical. The results are also compared with the standard probabilistic ranking algorithm BM25. Learning to rank strategies is also implemented, which shows substantial improvement in some of the result metrics. Experiments were performed on standard benchmarks like the TREC Microblog 2015, TREC RTS 2016, and TREC RTS 2017 datasets. Different variants of normal discounted cumulative gain, the standard official evaluation metric of TREC, nDCG-1, nDCG-0, and nDCG-p are used in this study. We also performed a comprehensive failure analysis on our experiments and identified key issues for improvement that can be addressed in the future.

Search Engine Reinforced Semi-Supervised Classification and Graph-Based Summarization of Microblogs

Exploring hypergraph-based semi-supervised ranking for query-oriented summarization

Re-ranking for Microblog Retrieval Via Multiple Graph Model

A Semi-Supervised Bayesian Network Model for Microblog Topic Classification.

Event Detection and Summarization on Microblogs

Continuous Summarization for Microblog Streams Based on Clustering.

Multimedia summarization for trending topics in microblogs.

An Efficient Approach to Summarizing Events from Microblogs

Towards Effective Event Detection, Tracking And Summarization On Microblog Data

Sentiment Classification of Financial Microblogs Through Automatic Text Summarization

Cross the Data Desert: Generating Textual-Visual Summary on the Evolutionary Microblog Stream

Social Media Opinion Summarization Using Emotion Cognition and Convolutional Neural Networks

SentiStory: Multi-Grained Sentiment Analysis and Event Summarization with Crowdsourced Social Media Data

A Simple Yet Effective Method for Summarizing Microblogging Users with Their Representative Tweets

Extended Search Method Based on a Semantic Hashtag Graph Combining Social and Conceptual Information

Microblog Summarization Using Paragraph Vector and Semantic Structure

Enhancing Keyword Suggestion of Web Search by Leveraging Microblog Data.

Design and analysis of microblog-based summarization system

Microblog Sentiment Classification Based on Supervised and Unsupervised Combination

A Study of Unsupervised Multi-document Sentiment Summarization Techniques Based on Graph Structure

Social Media Based Event Summarization by User–text–image Co-Clustering