Abstract:Obtaining a reliable estimate of the joint probability mass function (PMF) of a set of random variables from observed data is a significant objective in statistical signal processing and machine learning. Modelling the joint PMF as a tensor that admits a low-rank canonical polyadic decomposition (CPD) has enabled the development of efficient PMF estimation algorithms. However, these algorithms require the rank (model order) of the tensor to be specified beforehand. In real-world applications, the true rank is unknown. Therefore, an appropriate rank is usually selected from a candidate set either by observing validation errors or by computing various likelihood-based information criteria, a procedure which is computationally expensive for large datasets. This paper presents a novel Bayesian framework for estimating the joint PMF and automatically inferring its rank from observed data. We specify a Bayesian PMF estimation model and employ appropriate prior distributions for the model parameters, allowing for tuning-free rank inference via a single training run. We then derive a deterministic solution based on variational inference (VI) to approximate the posterior distributions of various model parameters. Additionally, we develop a scalable version of the VI-based approach by leveraging stochastic variational inference (SVI) to arrive at an efficient algorithm whose complexity scales sublinearly with the size of the dataset. Numerical experiments involving both synthetic data and real movie recommendation data illustrate the advantages of our VI and SVI-based methods in terms of estimation accuracy, automatic rank detection, and computational efficiency.

Bayesian rank penalization.

Bayesian Robust Principal Component Analysis with Adaptive Singular Value Penalty

Unified Low-Rank Matrix Estimate Via Penalized Matrix Least Squares Approximation

Bayesian Inference for Adaptive Low Rank and Sparse Matrix Estimation

Bayesian sparse multiple regression for simultaneous rank reduction and variable selection

Blm-Rank: A Bayesian Linear Method for Learning to Rank and Its Gpu Implementation

Double-Weighted Low-Rank Matrix Recovery Based on Rank Estimation.

Uncertainty Quantification in Bayesian Reduced-Rank Sparse Regressions

Bayesian Inference with the l1-ball Prior: Solving Combinatorial Problems with Exact Zeros

Bayesian rank-based hypothesis testing for the rank sum test, the signed rank test, and Spearman's ρ

Bayesian Rank-Clustering

Adaptive Rank Estimate in Robust Principal Component Analysis

Robust PCA via Nonconvex Rank Approximation

Rank method for partial functional linear regression models

Exact Penalization for Cardinality and Rank-Constrained Optimization Problems Via Partial Regularization

Sparse Bayesian Learning for Ranking.

Compound Poisson Processes, Latent Shrinkage Priors and Bayesian Nonconvex Penalization

Bayesian Estimation and Tuning-Free Rank Detection for Probability Mass Function Tensors

Generalized linear mixed model with bayesian rank likelihood

Bayesian Nonparametrics Meets Data-Driven Distributionally Robust Optimization

Low-Rank Tensor Completion: A Pseudo-Bayesian Learning Approach.