Abstract:A common approach to analyze a covariate-sample count matrix, an element of which represents how many times a covariate appears in a sample, is to factorize it under the Poisson likelihood. We show its limitation in capturing the tendency for a covariate present in a sample to both repeat itself and excite related ones. To address this limitation, we construct negative binomial factor analysis (NBFA) to factorize the matrix under the negative binomial likelihood, and relate it to a Dirichlet-multinomial distribution based mixed-membership model. To support countably infinite factors, we propose the hierarchical gamma-negative binomial process. By exploiting newly proved connections between discrete distributions, we construct two blocked and a collapsed Gibbs sampler that all adaptively truncate their number of factors, and demonstrate that the blocked Gibbs sampler developed under a compound Poisson representation converges fast and has low computational complexity. Example results show that NBFA has a distinct mechanism in adjusting its number of inferred factors according to the sample lengths, and provides clear advantages in parsimonious representation, predictive power, and computational complexity over previously proposed discrete latent variable models, which either completely ignore burstiness, or model only the burstiness of the covariates but not that of the factors.

A nonparametric variable clustering model

Strongly Hierarchical Factorization Machines and ANOVA Kernel Regression

Clustering and Prediction with Variable Dimension Covariates

A sparse factor model for clustering high‐dimensional longitudinal data

A Nonparametric Bayesian Technique for High-Dimensional Regression

Bayesian mixtures of common factor analyzers: Model, variational inference, and applications

Finite Mixtures of Multivariate Poisson-Log Normal Factor Analyzers for Clustering Count Data

DP-space: Bayesian Nonparametric Subspace Clustering with Small-variance Asymptotics.

High-dimensional variable clustering based on maxima of a weakly dependent random process

A variable clustering approach for overdispersed high-dimensional count data using a copula-based mixture model

Bayesian Statistical Inference for Factor Analysis Models with Clustered Data

Bayesian Clustering with Variable and Transformation Selections

Model-based Clustering of High-Dimensional Data: Variable Selection Versus Facet Determination

Sparse Bayesian Hierarchical Modeling of High-dimensional Clustering Problems

Parameter-wise co-clustering for high-dimensional data

Bayesian approach to clustering real value, categorical and network data: solution via variational methods

Identifiable and interpretable nonparametric factor analysis

Fast Nonparametric Clustering of Structured Time-Series

A new model for natural groupings in high-dimensional data

Clustering Multivariate Data using Factor Analytic Bayesian Mixtures with an Unknown Number of Components

Nonparametric Bayesian Negative Binomial Factor Analysis