Abstract:Relational topic models (RTMs) provide a probabilistic generative process to describe both the link structure and document contents for document networks, and they have shown promise on predicting network structures and discovering latent topic representations. However, existing RTMs have limitations in both the restricted model expressiveness and incapability of dealing with imbalanced network data. To expand the scope and improve the inference accuracy of RTMs, this paper presents three extensions: 1) unlike the common link likelihood with a diagonal weight matrix that allows the-same-topic interactions only, we generalize it to use a full weight matrix that captures all pairwise topic interactions and is applicable to asymmetric networks; 2) instead of doing standard Bayesian inference, we perform regularized Bayesian inference (RegBayes) with a regularization parameter to deal with the imbalanced link structure issue in real networks and improve the discriminative ability of learned latent representations; and 3) instead of doing variational approximation with strict mean-field assumptions, we present collapsed Gibbs sampling algorithms for the generalized relational topic models by exploring data augmentation without making restricting assumptions. Under the generic RegBayes framework, we carefully investigate two popular discriminative loss functions, namely, the logistic log-loss and the max-margin hinge loss. Experimental results on several real network datasets demonstrate the significance of these extensions on improving prediction performance.

Hierarchical relational models for document networks

Sparse Relational Topic Models for Document Networks.

Discriminative Relational Topic Models.

Hierarchical Topic Modeling with Nested Hierarchical Dirichlet Process

Scalable Models for Computing Hierarchies in Information Networks

Generalized Relational Topic Models with Data Augmentation.

On the Affinity, Rationality, and Diversity of Hierarchical Topic Modeling

Multi-relational Topic Model for Social Recommendation

Interactive Topic Modeling Based on Hierarchical Dirichlet Process

Topic Modeling Revisited: A Document Graph-based Neural Network Perspective

A Topic Model for Hierarchical Documents

Topic Modeling with Network Regularization

Graph Neural Collaborative Topic Model for Citation Recommendation

Learning Topic Hierarchies by Tree-Directed Latent Variable Models

Modeling Documents with Event Model

Hierarchical Infinite Relational Model

Neural Topic Modeling by Incorporating Document Relationship Graph

RankTopic: Ranking Based Topic Modeling

Utilizing Recurrent Neural Network for Topic Discovery in Short Text Scenarios

Scalable Training of Hierarchical Topic Models

A network approach to topic models