Abstract:Obtaining a reliable estimate of the joint probability mass function (PMF) of a set of random variables from observed data is a significant objective in statistical signal processing and machine learning. Modelling the joint PMF as a tensor that admits a low-rank canonical polyadic decomposition (CPD) has enabled the development of efficient PMF estimation algorithms. However, these algorithms require the rank (model order) of the tensor to be specified beforehand. In real-world applications, the true rank is unknown. Therefore, an appropriate rank is usually selected from a candidate set either by observing validation errors or by computing various likelihood-based information criteria, a procedure which is computationally expensive for large datasets. This paper presents a novel Bayesian framework for estimating the joint PMF and automatically inferring its rank from observed data. We specify a Bayesian PMF estimation model and employ appropriate prior distributions for the model parameters, allowing for tuning-free rank inference via a single training run. We then derive a deterministic solution based on variational inference (VI) to approximate the posterior distributions of various model parameters. Additionally, we develop a scalable version of the VI-based approach by leveraging stochastic variational inference (SVI) to arrive at an efficient algorithm whose complexity scales sublinearly with the size of the dataset. Numerical experiments involving both synthetic data and real movie recommendation data illustrate the advantages of our VI and SVI-based methods in terms of estimation accuracy, automatic rank detection, and computational efficiency.

Bayesian rank-based hypothesis testing for the rank sum test, the signed rank test, and Spearman's ρ

Bayesian Rank-Clustering

Bayesian Hypothesis Tests Using Nonparametric Statistics

Bayesian Estimation of Kendall's tau Using a Latent Normal Approach

Generalized linear mixed model with bayesian rank likelihood

Bayesian inferences on uncertain ranks and orderings: Application to ranking players and lineups

Bayesian rank penalization.

Exact and efficient multivariate two-sample tests through generalized linear rank statistics

Bayesian Estimation and Tuning-Free Rank Detection for Probability Mass Function Tensors

Bayesian Aggregation of Order-Based Rank Data

Another Argument in Favour of Wilcoxon's Signed Rank Test

A puzzle of proportions: Two popular Bayesian tests can yield dramatically different conclusions

Rank-Based Procedures in Factorial Designs: Hypotheses about Nonparametric Treatment Effects

Optimal rank-based testing for principal components

Bayesian ranking and selection with applications to field studies, economic mobility, and forecasting

Revealing subgroup structure in ranked data using a Bayesian WAND

On Rosenbaum's Rank-based Matching Estimator

Bayesian tests of two proportions: A tutorial with R and JASP

Towards a theory for testing statistical hypothesis: Multivariate mean with nuisance covariance matrix

Nonparametric Rank Test for Nonlinearity Detection

A simple consistent Bayes factor for testing the Kendall rank correlation coefficient