Abstract:Probabilistic topic models are unsupervised generative models which model document content as a two-step generation process, that is, documents are observed as mixtures of latent concepts or topics, while topics are probability distributions over vocabulary words. Recently, a significant research effort has been invested into transferring the probabilistic topic modeling concept from monolingual to multilingual settings. Novel topic models have been designed to work with parallel and comparable texts. We define multilingual probabilistic topic modeling (MuPTM) and present the first full overview of the current research, methodology, advantages and limitations in MuPTM. As a representative example, we choose a natural extension of the omnipresent LDA model to multilingual settings called bilingual LDA (BiLDA). We provide a thorough overview of this representative multilingual model from its high-level modeling assumptions down to its mathematical foundations. We demonstrate how to use the data representation by means of output sets of (i) per-topic word distributions and (ii) per-document topic distributions coming from a multilingual probabilistic topic model in various real-life cross-lingual tasks involving different languages, without any external language pair dependent translation resource: (1) cross-lingual event-centered news clustering, (2) cross-lingual document classification, (3) cross-lingual semantic similarity, and (4) cross-lingual information retrieval. We also briefly review several other applications present in the relevant literature, and introduce and illustrate two related modeling concepts: topic smoothing and topic pruning. In summary, this article encompasses the current research in multilingual probabilistic topic modeling. By presenting a series of potential applications, we reveal the importance of the language-independent and language pair independent data representations by means of MuPTM. We provide clear directions for future research in the field by providing a systematic overview of how to link and transfer aspect knowledge across corpora written in different languages via the shared space of latent cross-lingual topics, that is, how to effectively employ learned per-topic word distributions and per-document topic distributions of any multilingual probabilistic topic model in various cross-lingual applications.

Combining Thesaurus Knowledge and Probabilistic Topic Models

Incorporating Probabilistic Knowledge into Topic Models.

Short Text Understanding by Leveraging Knowledge into Topic Model.

Incorporating Knowledge Graph Embeddings into Topic Modeling

Mining Coherent Topics in Documents Using Word Embeddings and Large-Scale Text Data

Concept over Time: the Combination of Probabilistic Topic Model with Wikipedia Knowledge.

Source-LDA: Enhancing probabilistic topic models using prior knowledge sources

Integration of Neural Embeddings and Probabilistic Models in Topic Modeling

Probabilistic Topic Modeling for Comparative Analysis of Document Collections

Probabilistic Topic Modeling in Multilingual Settings: an Overview of Its Methodology and Applications.

Probabilistic Topic Modelling with Transformer Representations

Topic-based mixture language modelling

A two-stage hybrid probabilistic topic model for refining image annotation

Parsimonious Topic Models with Salient Word Discovery

Probabilistic Non-Negative Matrix Factorization and Its Robust Extensions for Topic Modeling.

Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning

Statistical Word Sense Aware Topic Models

Grounding Topic Models with Knowledge Bases.

Topic Signature Language Models for Ad hoc Retrieval

Topic Models Incorporating Statistical Word Senses

Nonparametric Topic Modeling with Neural Inference