Abstract:Topic modeling provides a powerful way to analyze the content of a collection of documents. It has become a popular tool in many research areas, such as text mining, information retrieval, natural language processing, and other related fields. In real-world applications, however, the usefulness of topic modeling is limited due to scalability issues. Scaling to larger document collections via parallelization is an active area of research, but most solutions require drastic steps, such as vastly reducing input vocabulary. In this article we introduce Regularized Latent Semantic Indexing (RLSI)---including a batch version and an online version, referred to as batch RLSI and online RLSI, respectively---to scale up topic modeling. Batch RLSI and online RLSI are as effective as existing topic modeling techniques and can scale to larger datasets without reducing input vocabulary. Moreover, online RLSI can be applied to stream data and can capture the dynamic evolution of topics. Both versions of RLSI formalize topic modeling as a problem of minimizing a quadratic loss function regularized by &ell;1 and/or &ell;2 norm. This formulation allows the learning process to be decomposed into multiple suboptimization problems which can be optimized in parallel, for example, via MapReduce. We particularly propose adopting &ell;1 norm on topics and &ell;2 norm on document representations to create a model with compact and readable topics and which is useful for retrieval. In learning, batch RLSI processes all the documents in the collection as a whole, while online RLSI processes the documents in the collection one by one. We also prove the convergence of the learning of online RLSI. Relevance ranking experiments on three TREC datasets show that batch RLSI and online RLSI perform better than LSI, PLSI, LDA, and NMF, and the improvements are sometimes statistically significant. Experiments on a Web dataset containing about 1.6 million documents and 7 million terms, demonstrate a similar boost in performance.

An Improved Regularized Latent Semantic Indexing with L1/2 Regularization and Non-negative Constraints

Regularized Latent Semantic Indexing: A New Approach to Large-Scale Topic Modeling

Efficient Probabilistic Latent Semantic Analysis with Sparsity Control

STMLRC: Sparse Topic Model with Low Rank Constraint

Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning

Learning Semantic Textual Similarity Via Topic-informed Discrete Latent Variables

Sparse Relational Topic Models for Document Networks.

Sparse online topic models

Semantic Augmented Topic Model over Short Text

Learning to rank with (a lot of) word features

A Simplified Latent Semantic Indexing Approach for Multi-Linguistic Information Retrieval.

Enhanced Short Text Modeling: Leveraging Large Language Models for Topic Refinement

Improving Latent Semantic Indexing With Concepts Mapping Based On Domain Ontology

Hierarchy-Regularized Latent Semantic Indexing

Regularized Locality Preserving Indexing Via Spectral Regression

3D Model Retrieval Based on Latent Semantic Linear Embedding

Modeling Hidden Topics with Dual Local Consistency for Image Analysis

Latent Space Regularization for Recommender Systems.

Graph Regularized Non-negative Matrix Factorization with Long-tail Constraint

Finding Semantically Valid and Relevant Topics by Association-Based Topic Selection Model

LATENT SEMANTIC INDEXING (LSI) AND ITS APPLICATIONS IN CHINESE TEXT PROCESSING