Collecting, Managing and Analyzing Social Networking Data Effectively

Zhongying Zhao,Zheng Feng,Yong Zhang,Li Ning,Jiancong Fan,Shengzhong Feng
DOI: https://doi.org/10.1109/fskd.2015.7382191
2015-01-01
Abstract:Social network analysis is a hot topic in recent years. However, how to collect, manage and analyze those large volume data effectively, is a very important problem. In this paper, we describe a system to collect, manage and mine social media data. Taking Tencent microblogs as an example, we implement a social media data collecting module based on Hadoop. In order to process data effectively and efficiently, we design a storm based managing and preprocessing system. As to the social media data analysis, we conduct topic detection based on LDA algorithm. We also propose a method to calculate the hot degree for each topic. In order to analyze the topic dynamics, we present a topic tracking method. Furthermore, we give an integrated method, which combines emotion and dictionary to analyze the sentiment of each topic. The Web-based result visualization are also implemented.
What problem does this paper attempt to address?