Stream-based live public opinion monitoring approach with adaptive probabilistic topic model

Kun Ma,Ziqiang Yu,Ke Ji,Bo Yang
DOI: https://doi.org/10.1007/s00500-018-3391-7
IF: 3.732
2018-07-17
Soft Computing
Abstract:Public opinion monitoring, also known as first story detection, is defined within the topic detection and tracking on a particular Internet news event. Generally, it is used to find news propagation. Traditional method adopts text matching to address opinion monitoring. But it has some limitations such as hidden and latent topic discovery and incorrect relevance ranking of matching results on large-scale data. In this paper, we propose three solutions to live public opinion monitoring: simple keyword computing and matching, simple probabilistic topic computing and matching, and stream-based live probabilistic topic computing and matching. We point out the disadvantages of the first two solutions such as semantic matching and low efficiency on timely big data. Stream-based real-time topic computing and topic matching with query-time document and field boosting are proposed to make substantial improvements. Finally, our topic computing and matching experiments with crawled historical Netease news records show that our approaches are effective and efficient.
computer science, artificial intelligence, interdisciplinary applications
What problem does this paper attempt to address?