Abstract:There is an explosion of data, documents, and other content, and people require tools to analyze and interpret these, tools to turn the content into information and knowledge. Topic modeling have been developed to solve these problems. Topic models such as LDA [Blei et. al. 2003] allow salient patterns in data to be extracted automatically. When analyzing texts, these patterns are called topics. Among numerous extensions of LDA, few of them can reliably analyze multiple groups of documents and extract topic similarities. Recently, the introduction of differential topic modeling (SPDP) [Chen et. al. 2012] performs uniformly better than many topic models in a discriminative setting. There is also a need to improve the sampling speed for topic models. While some effort has been made for distributed algorithms, there is no work currently done using graphical processing units (GPU). Note the GPU framework has already become the most cost-efficient platform for many problems. In this thesis, I propose and implement a scalable multi-GPU distributed parallel framework which approximates SPDP. Through experiments, I have shown my algorithms have a gain in speed of about 50 times while being almost as accurate, with only one single cheap laptop GPU. Furthermore, I have shown the speed improvement is sublinearly scalable when multiple GPUs are used, while fairly maintaining the accuracy. Therefore on a medium-sized GPU cluster, the speed improvement could potentially reach a factor of a thousand. Note SPDP is just a representative of other extensions of LDA. Although my algorithm is implemented to work with SPDP, it is designed to be a general enough to work with other topic models. The speed-up on smaller collections (i.e., 1000s of documents), means that these more complex LDA extensions could now be done in real-time, thus opening up a new way of using these LDA models in industry.

Parallel Inference for Latent Dirichlet Allocation on Graphics Processing Units.

GLDA: Parallel Gibbs Sampling for Latent Dirichlet Allocation on GPU.

CuLDA_CGS: Solving Large-scale LDA Problems on GPUs

CuLDA_CGS

CuLDA: Solving Large-scale LDA Problems on GPUs

Accelerating Topic Model Training on a Single Machine

Parallel Data Mining on CUDA-enabled Graphics Processing Unit(GPU)

Multi-GPU Distributed Parallel Bayesian Differential Topic Modelling

Parallel L-BFGS-B Algorithm on GPU.

PLDA+: Parallel latent dirichlet allocation with data placement and pipeline processing

Parallel data mining techniques on Graphics Processing Unit with Compute Unified Device Architecture (CUDA)

Efficient Parallel Implementation of the Lattice Boltzmann Method on Large Clusters of Graphic Processing Units

Large-scale Paralleled Sparse Principal Component Analysis.

A New Method To Parallel Implementation For Batching Vast Small-Scale Computing Tasks Based On Gpu

CuMF_SGD: Parallelized Stochastic Gradient Descent for Matrix Factorization on GPUs.

Parallel Data Mining on Graphics Processors

Efficient Gradient Boosted Decision Tree Training on GPUs

GPregel: A GPU-Based Parallel Graph Processing Model

An Incremental Iterative Acceleration Architecture in Distributed Heterogeneous Environments With GPUs for Deep Learning

Efficient Knowledge Graph Embedding Training Framework with Multiple GPUs

ParvaGPU: Efficient Spatial GPU Sharing for Large-Scale DNN Inference in Cloud Environments