A network approach to topic models

Martin Gerlach,Tiago P. Peixoto,Eduardo G. Altmann

DOI: https://doi.org/10.1126/sciadv.aaq1360

IF: 13.6

2018-07-06

Science Advances

Abstract:A new approach to topic models finds topics through community detection in word-document networks.

multidisciplinary sciences

What problem does this paper attempt to address?

Adding Community and Dynamic to Topic Models.

Daifeng Li,Ying Ding,Xin Shuai,Johan Bollen,Jie Tang,Shanshan Chen,Jiayi Zhu,Guilherme Rocha

DOI: https://doi.org/10.1016/j.joi.2011.11.004

IF: 3.7

2012-01-01

Journal of Informetrics

Abstract:The detection of communities in large social networks is receiving increasing attention in a variety of research areas. Most existing community detection approaches focus on the topology of social connections (e.g., coauthor, citation, and social conversation) without considering their topic and dynamic features. In this paper, we propose two models to detect communities by considering both topic and dynamic features. First, the Community Topic Model (CTM) can identify communities sharing similar topics. Second, the Dynamic CTM (DCTM) can capture the dynamic features of communities and topics based on the Bernoulli distribution that leverages the temporal continuity between consecutive timestamps. Both models were tested on two datasets: ArnetMiner and Twitter. Experiments show that communities with similar topics can be detected and the co-evolution of communities and topics can be observed by these two models, which allow us to better understand the dynamic features of social networks and make improved personalized recommendations. (C) 2011 Elsevier Ltd. All rights reserved.
Topic Modeling with Network Regularization

Qiaozhu Mei,Deng Cai,Duo Zhang,ChengXiang Zhai

DOI: https://doi.org/10.1145/1367497.1367512

2008-01-01

Abstract:In this paper, we formally define the problem of topic modeling with network structure (TMN). We propose a novel solution to this problem, which regularizes a statistical topic model with a harmonic regularizer based on a graph structure in the data. The proposed method bridges topic modeling and social network analysis, which leverages the power of both statistical topic models and discrete regularization. The output of this model well summarizes topics in text, maps a topic on the network, and discovers topical communities. With concrete selection of a topic model and a graph-based regularizer, our model can be applied to text mining problems such as author-topic analysis, community discovery, and spatial text mining. Empirical experiments on two different genres of data show that our approach is effective, which improves text-oriented methods as well as network-oriented methods. The proposed model is general; it can be applied to any text collections with a mixture of topics and an associated network structure.
Topic Modeling Using Distributed Word Embeddings

Ramandeep S Randhawa,Parag Jain,Gagan Madan

DOI: https://doi.org/10.48550/arXiv.1603.04747

2016-03-16

Abstract:We propose a new algorithm for topic modeling, Vec2Topic, that identifies the main topics in a corpus using semantic information captured via high-dimensional distributed word embeddings. Our technique is unsupervised and generates a list of topics ranked with respect to importance. We find that it works better than existing topic modeling techniques such as Latent Dirichlet Allocation for identifying key topics in user-generated content, such as emails, chats, etc., where topics are diffused across the corpus. We also find that Vec2Topic works equally well for non-user generated content, such as papers, reports, etc., and for small corpora such as a single-document.

Computation and Language
Term-community-based topic detection with variable resolution

Andreas Hamm,Simon Odrowski

DOI: https://doi.org/10.3390/info12060221

2021-07-24

Abstract:Network-based procedures for topic detection in huge text collections offer an intuitive alternative to probabilistic topic models. We present in detail a method that is especially designed with the requirements of domain experts in mind. Like similar methods, it employs community detection in term co-occurrence graphs, but it is enhanced by including a resolution parameter that can be used for changing the targeted topic granularity. We also establish a term ranking and use semantic word-embedding for presenting term communities in a way that facilitates their interpretation. We demonstrate the application of our method with a widely used corpus of general news articles and show the results of detailed social-sciences expert evaluations of detected topics at various resolutions. A comparison with topics detected by Latent Dirichlet Allocation is also included. Finally, we discuss factors that influence topic interpretation.

Computation and Language
Topic Modeling Revisited: A Document Graph-based Neural Network Perspective

Dazhong Shen,Chuan Qin,Chao Wang,Zheng Dong,Hengshu Zhu,Hui Xiong

2021-01-01

Abstract:Most topic modeling approaches are based on the bag-of-words assumption, where each word is required to be conditionally independent in the same document. As a result, both of the generative story and the topic formulation have totally ignored the semantic dependency among words, which is important for improving the semantic comprehension and model interpretability. To this end, in this paper, we revisit the task of topic modeling by transforming each document into a directed graph with word dependency as edges between word nodes, and develop a novel approach, namely Graph Neural Topic Model (GNTM). Specifically, in GNTM, a well-defined probabilistic generative story is designed to model both the graph structure and word sets with multinomial distributions on the vocabulary and word dependency edge set as the topics. Meanwhile, a Neural Variational Inference (NVI) approach is proposed to learn our model with graph neural networks to encode the document graphs. Besides, we theoretically demonstrate that Latent Dirichlet Allocation (LDA) can be derived from GNTM as a special case with similar objective functions. Finally, extensive experiments on four benchmark datasets have clearly demonstrated the effectiveness and interpretability of GNTM compared with state-of-the-art baselines.
Modeling Community Structure and Topics in Dynamic Text Networks

Teague R. Henry,David Banks,Derek Owens-Oas,Christine Chai

DOI: https://doi.org/10.1007/s00357-018-9289-3

IF: 1.333

2019-01-22

Journal of Classification

Abstract:The last decade has seen great progress in both dynamic network modeling and topic modeling. This paper draws upon both areas to create a bespoke Bayesian model applied to a dataset consisting of the top 467 US political blogs in 2012, their posts over the year, and their links to one another. Our model allows dynamic topic discovery to inform the latent network model and the network structure to facilitate topic identification. Our results find complex community structure within this set of blogs, where community membership depends strongly upon the set of topics in which the blogger is interested. We examine the time varying nature of the Sensational Crime topic, as well as the network properties of the Election News topic, as notable and easily interpretable empirical examples.

mathematics, interdisciplinary applications,psychology, mathematical
Community-based Topic Modeling for Social Tagging.

Daifeng Li,Bing He,Ying Ding,Jie Tang,Cassidy R. Sugimoto,Zheng Qin,Erjia Yan,Juanzi Li,Tianxi Dong

DOI: https://doi.org/10.1145/1871437.1871673

2010-01-01

Abstract:Exploring community is fundamental for uncovering the connections between structure and function of complex networks and for practical applications in many disciplines such as biology and sociology. In this paper, we propose a TTR-LDA-Community model which combines the Latent Dirichlet Allocation model (LDA) and the Girvan-Newman community detection algorithm with an inference mechanism. The model is then applied to data from Delicious, a popular social tagging system, over the time period of 2005-2008. Our results show that 1) users in the same community tend to be interested in similar set of topics in all time periods; and 2) topics may divide into several sub-topics and scatter into different communities over time. We evaluate the effectiveness of our model and show that the TTR-LDA-Community model is meaningful for understanding communities and outperforms TTR-LDA and LDA models in tag prediction.
Graph-Community Detection for Cross-Document Topic Segment Relationship Identification

Pedro Mota,Maxine Eskenazi,Luisa Coheur

DOI: https://doi.org/10.48550/arXiv.1606.04081

2016-06-14

Abstract:In this paper we propose a graph-community detection approach to identify cross-document relationships at the topic segment level. Given a set of related documents, we automatically find these relationships by clustering segments with similar content (topics). In this context, we study how different weighting mechanisms influence the discovery of word communities that relate to the different topics found in the documents. Finally, we test different mapping functions to assign topic segments to word communities, determining which topic segments are considered equivalent. By performing this task it is possible to enable efficient multi-document browsing, since when a user finds relevant content in one document we can provide access to similar topics in other documents. We deploy our approach in two different scenarios. One is an educational scenario where equivalence relationships between learning materials need to be found. The other consists of a series of dialogs in a social context where students discuss commonplace topics. Results show that our proposed approach better discovered equivalence relationships in learning material documents and obtained close results in the social speech domain, where the best performing approach was a clustering technique.

Computation and Language,Information Retrieval,Social and Information Networks
Twitter-Network Topic Model: A Full Bayesian Treatment for Social Network and Text Modeling

Kar Wai Lim,Changyou Chen,Wray Buntine

DOI: https://doi.org/10.48550/arXiv.1609.06791

2016-09-22

Computation and Language

Abstract:Twitter data is extremely noisy -- each tweet is short, unstructured and with informal language, a challenge for current topic modeling. On the other hand, tweets are accompanied by extra information such as authorship, hashtags and the user-follower network. Exploiting this additional information, we propose the Twitter-Network (TN) topic model to jointly model the text and the social network in a full Bayesian nonparametric way. The TN topic model employs the hierarchical Poisson-Dirichlet processes (PDP) for text modeling and a Gaussian process random function model for social network modeling. We show that the TN topic model significantly outperforms several existing nonparametric models due to its flexibility. Moreover, the TN topic model enables additional informative inference such as authors' interests, hashtag analysis, as well as leading to further applications such as author recommendation, automatic topic labeling and hashtag suggestion. Note our general inference framework can readily be applied to other topic models with embedded PDP nodes.
A Local Context-Aware LDA Model for Topic Modeling in a Document Network

Yang Liu,Songhua Xu

DOI: https://doi.org/10.1002/asi.23822

2017-01-01

Journal of the Association for Information Science and Technology

Abstract:With the rapid development of the Internet and its applications, growing volumes of documents increasingly become interconnected to form large-scale document networks. Accordingly, topic modeling in a network of documents has been attracting continuous research attention. Most of the existing network-based topic models assume that topics in a document are influenced by its directly linked neighbouring documents in a document network and overlook the potential influence from indirectly linked ones. The existing work also has not carefully modeled variations of such influence among neighboring documents. Recognizing these modeling limitations, this paper introduces a novel Local Context-Aware LDA Model LC-LDA, which is capable of observing a local context comprising a rich collection of documents that may directly or indirectly influence the topic distributions of a target document. The proposed model can also differentiate the respective influence of each document in the local context on the target document according to both structural and temporal relationships between the two documents. The proposed model is extensively evaluated through multiple document clustering and classification tasks conducted over several large-scale document sets. Evaluation results clearly and consistently demonstrate the effectiveness and superiority of the new model with respect to several state-of-the-art peer models.
Hierarchical Community Discovery for Social Networks Based on Probabilistic Topic Model

BI Juan,QIN Zhi-guang

DOI: https://doi.org/10.3969/j.issn.1001-0548.2014.06.018

2014-01-01

Abstract:The traditional community discovery algorithms are generally based on the link structure of a given social network, they lack of consideration of user’s interests and the hierarchical structure of community. In this paper, a novel PAM (Pachinko Allocation Model) probabilistic generative model is proposed to detect latent hierarchical communities based on the user interests and their social relationships. The joint model of topic modeling and community discovery can capture the correlation among multiple communities and their hierarchical structure. Experiments on real-world dataset have confirmed the feasibility and effectiveness of the proposed algorithm.
A Relationship Strength-Aware Topic Model for Communities Discovery in Online Social Networks

Juan Bi,Jia Huang,Zhiguang Qin

DOI: https://doi.org/10.1007/978-3-642-41674-3_101

2014-01-01

Abstract:Automatic discovering latent communities of users from observed textual content and their relationships is vital for understanding the cooperation and interaction patterns of users on large scale. In this paper, a novel probabilistic generative model was proposed to detect latent communities in a social network based on semantic information and the social relationships between users. In this model, it was assumed that users from the same community tend to share similar interests, and those who engage in common topics should be closely connected to each other on the topology structure of the social network. Users can have multiple interests and participate in multiple communities. Further, heterogeneous relationship strength was used in this paper to improve community discovery. The research indicated that the probabilistic generative model present in this paper has a good capability of discovering meaningful communities and topics on real-world data from Twitter.
A Methodology for Integrating Network Theory and Topic Modeling and its Application to Innovation Diffusion

Jana Diesner,Kathleen M. Carley

DOI: https://doi.org/10.1109/socialcom.2010.106

2010-08-01

Abstract:Text data pertaining to socio-technical networks often are analyzed separately from relational data, or are reduced to the fact and strength of the flow of information between nodes. Disregarding the content of text data for network analysis can limit our understanding of the effects of language use in networks. We present a computational and interdisciplinary methodology that addresses this limitation by combining theory from socio-linguistics with social network analysis and machine learning based text mining: we use network analysis to identify groups of individuals who assume the theoretically grounded roles of change agents and preservation agents. People in these roles differ in their motivation and capability to induce and adopt change in a network. Topic modeling is then constrained to the texts authored by people in these roles. We apply this methodology to a public dataset of about 55,000 research proposals that were granted funding. Our results suggest that the people per role differ in the research domains they work on and the strength of association with those domains that both roles are involved with, but are similar with respect to fulfilling the task or additional role of being a project manager.
Topic Modeling based on Keywords and Context

Johannes Schneider

DOI: https://doi.org/10.48550/arXiv.1710.02650

2018-02-04

Abstract:Current topic models often suffer from discovering topics not matching human intuition, unnatural switching of topics within documents and high computational demands. We address these concerns by proposing a topic model and an inference algorithm based on automatically identifying characteristic keywords for topics. Keywords influence topic-assignments of nearby words. Our algorithm learns (key)word-topic scores and it self-regulates the number of topics. Inference is simple and easily parallelizable. Qualitative analysis yields comparable results to state-of-the-art models (eg. LDA), but with different strengths and weaknesses. Quantitative analysis using 9 datasets shows gains in terms of classification accuracy, PMI score, computational performance and consistency of topic assignments within documents, while most often using less topics.

Computation and Language,Information Retrieval,Machine Learning
Topic-level social network search.

Jie Tang,Sen Wu,Bo Gao,Yang Wan

DOI: https://doi.org/10.1145/2020408.2020532

2011-01-01

Abstract:We study the problem of topic-level social network search, which aims to find who are the most influential users in a network on a specific topic and how the influential users connect with each other. We employ a topic model to find topical aspects of each user and a retrieval method to identify influential users by combining the language model and the topic model. An influence maximization algorithm is then presented to find the sub network that closely connects the influential users. Two demonstration systems have been developed and are online available. Empirical analysis based on the user's viewing time and the number of clicks validates the proposed methodologies.
Topic Network Analysis Based on Co-Occurrence Time Series Clustering

Weibin Lin,Xianli Wu,Zhengwei Wang,Xiaoji Wan,Hailin Li

DOI: https://doi.org/10.3390/math10162846

IF: 2.4

2022-08-11

Mathematics

Abstract:Traditional topic research divides similar topics into the same cluster according to clustering or classification from the perspective of users, which ignores the deep relationship within and between topics. In this paper, topic analysis is achieved from the perspective of the topic network. Based on the initial core topics obtained by the keyword importance and affinity propagation clustering, co-occurrence time series between topics are constructed according to time sequence and topic frequency. Subsequence segments of each topic co-occurrence time series are divided by sliding windows, and the similarity between subsequence segments is calculated. Based on the topic similarity matrix, the topic network is constructed. The topic network is divided according to the community detection algorithm, which realizes the topic re-clustering and reveals the deep relationship between topics in fine-grained. The results show there is no relationship between topic center representation and keyword popularity, and topics with a wide range of concepts are more likely to become topic network centers. The proposed approach takes into account the influence of time factors on topic analysis, which not only expands the analysis in the field of topic research but also improves the quality of topic research.

mathematics
A Bimodal Network Approach to Model Topic Dynamics

Luigi Di Caro,Marco Guerzoni,Massimiliano Nuccio,Giovanni Siragusa

DOI: https://doi.org/10.48550/arXiv.1709.09373

2017-09-27

Abstract:This paper presents an intertemporal bimodal network to analyze the evolution of the semantic content of a scientific field within the framework of topic modeling, namely using the Latent Dirichlet Allocation (LDA). The main contribution is the conceptualization of the topic dynamics and its formalization and codification into an algorithm. To benchmark the effectiveness of this approach, we propose three indexes which track the transformation of topics over time, their rate of birth and death, and the novelty of their content. Applying the LDA, we test the algorithm both on a controlled experiment and on a corpus of several thousands of scientific papers over a period of more than 100 years which account for the history of the economic thought.

Computation and Language,General Economics
Detecting Communities with Multiplex Semantics by Distinguishing Background, General, and Specialized Topics

Di Jin,Kunzeng Wang,Ge Zhang,Pengfei Jiao,Dongxiao He,Francoise Fogelman-Soulie,Xin Huang

DOI: https://doi.org/10.1109/tkde.2019.2937298

IF: 9.235

2019-01-01

IEEE Transactions on Knowledge and Data Engineering

Abstract:Finding semantic communities using network topology and contents together is a hot topic in community detection. Existing methods often use word attributes in an indiscriminate way to help finding communities. Through analysis we find that, words in networked contents often embody a hierarchical semantic structure. Some words reflect a background topic of the whole network with all communities, some imply the high-level general topic covering several topic-related communities, and some imply the high-resolution specialized topic to describe each community. Ignoring such semantic structures often leads to defects in depicting networked contents where deep semantics are not fully utilized. To solve this problem, we propose a new Bayesian probabilistic model. By distinguishing words from either a background topic or some two-level topics (i.e., general and specialized topics), this model not only better utilizes the networked contents to help finding communities, but also provides a clearer multiplex semantic community interpretation. We then give an efficient variational algorithm for model inference. The superiority of this new approach is demonstrated by comparing with ten state-of-the-art methods on nine real networks and an artificial benchmark. A case study is further provided to show its strong ability in deep semantic interpretation of communities.
Nonparametric Models for Characterizing the Topical Communities in Social Network

Ziqi Liu,Qinghua Zheng,Fei Wang,Buyue Qian

DOI: https://doi.org/10.1016/j.neucom.2016.07.053

IF: 6

2016-01-01

Neurocomputing

Abstract:Topical communities have shown useful tools for characterizing social networks. However data in social networks often come as streams, i.e., both text content (e.g., emails, user postings) and network structure (e.g., user friendship) evolve over time. We propose two nonparametric statistic models where infinite latent community variables coupled with infinite latent topic variables. The temporal dependencies between variables across epochs are modeled via a rich-gets-richer scheme. We focus on characterizing three dynamic aspects in social streams: the number of communities or topics changes (e.g., new communities or topics are born and old ones die out); the popularity of communities or topics evolves; the semantics such as community topic distribution, community participant distribution and topic word distribution drift. Furthermore, we develop an effective online posterior inference algorithm for the models, which is concordant with the online nature of social streams. Experiments using real-world data show the effectiveness of our model at discovering the dynamic topical communities in social streams.
Network-based Topic Structure Visualization

Yeseul Jeon,Jina Park,Ick Hoon Jin,Dongjun Chungc

2024-01-31

Abstract:In the real world, many topics are inter-correlated, making it challenging to investigate their structure and relationships. Understanding the interplay between topics and their relevance can provide valuable insights for researchers, guiding their studies and informing the direction of research. In this paper, we utilize the topic-words distribution, obtained from topic models, as item-response data to model the structure of topics using a latent space item response model. By estimating the latent positions of topics based on their distances toward words, we can capture the underlying topic structure and reveal their relationships. Visualizing the latent positions of topics in Euclidean space allows for an intuitive understanding of their proximity and associations. We interpret relationships among topics by characterizing each topic based on representative words selected using a newly proposed scoring scheme. Additionally, we assess the maturity of topics by tracking their latent positions using different word sets, providing insights into the robustness of topics. To demonstrate the effectiveness of our approach, we analyze the topic composition of COVID-19 studies during the early stage of its emergence using biomedical literature in the PubMed database. The software and data used in this paper are publicly available at

Applications,Human-Computer Interaction,Information Retrieval

A network approach to topic models

Adding Community and Dynamic to Topic Models.

Topic Modeling with Network Regularization

Topic Modeling Using Distributed Word Embeddings

Term-community-based topic detection with variable resolution

Topic Modeling Revisited: A Document Graph-based Neural Network Perspective

Modeling Community Structure and Topics in Dynamic Text Networks

Community-based Topic Modeling for Social Tagging.

Graph-Community Detection for Cross-Document Topic Segment Relationship Identification

Twitter-Network Topic Model: A Full Bayesian Treatment for Social Network and Text Modeling

A Local Context-Aware LDA Model for Topic Modeling in a Document Network

Hierarchical Community Discovery for Social Networks Based on Probabilistic Topic Model

A Relationship Strength-Aware Topic Model for Communities Discovery in Online Social Networks

A Methodology for Integrating Network Theory and Topic Modeling and its Application to Innovation Diffusion

Topic Modeling based on Keywords and Context

Topic-level social network search.

Topic Network Analysis Based on Co-Occurrence Time Series Clustering

A Bimodal Network Approach to Model Topic Dynamics

Detecting Communities with Multiplex Semantics by Distinguishing Background, General, and Specialized Topics

Nonparametric Models for Characterizing the Topical Communities in Social Network

Network-based Topic Structure Visualization