Abstract:The cumulative shrinkage process is an increasing shrinkage prior that can be employed within models in which additional terms are supposed to play a progressively negligible role. A natural application is to Gaussian factor models, where such a process has proved effective in inducing parsimonious representations while providing accurate inference on the data covariance matrix. The cumulative shrinkage process came with an adaptive Gibbs sampler that tunes the number of latent factors throughout iterations, which makes it faster than the non-adaptive Gibbs sampler. In this work we propose a variational algorithm for Gaussian factor models endowed with a cumulative shrinkage process. Such a strategy provides comparable inference with respect to the adaptive Gibbs sampler and further reduces runtime
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to perform variational Bayesian inference using the cumulative shrinkage process in the Gaussian factor model to improve computational efficiency and model sparsity. Specifically, the goals of the paper include:
1. **Improve computational efficiency**: By proposing a variational Bayesian algorithm for the Gaussian factor model, this algorithm further reduces the running time while maintaining inference performance comparable to that of the adaptive Gibbs sampler. This enables the algorithm to better handle large - scale datasets and meet the needs of the big data era.
2. **Enhance model sparsity**: Using the cumulative shrinkage process as a prior, it encourages the latent factors in the model to gradually become unimportant, thereby achieving a parsimonious representation of the model. This prior helps to automatically determine the number of active latent factors and avoid over - fitting.
3. **Simplify algorithm implementation**: By introducing the auxiliary variable \(z_h\), the derivation process of the variational algorithm is simplified, making the algorithm easier to implement and optimize.
### Main contributions of the paper
- **Variational Bayesian algorithm**: A variational Bayesian algorithm for the Gaussian factor model based on the cumulative shrinkage process is proposed, which significantly reduces the computational time while maintaining inference accuracy.
- **Performance comparison**: Through experiments on real - world datasets, it is verified that the proposed variational Bayesian algorithm has inference performance comparable to that of the adaptive Gibbs sampler, but with a much shorter running time.
- **Theoretical basis**: The definition of the cumulative shrinkage process and its application in the Gaussian factor model are introduced in detail, and the specific steps of the variational Bayesian algorithm are derived.
### Formula summary
- **Definition of the cumulative shrinkage process**:
\[
(\theta_h|\pi_h)\sim(1 - \pi_h)P_0+\pi_hP_{\infty},\quad(h = 1,\ldots,H)
\]
where \(\pi_h=\sum_{l = 1}^h\omega_l\), \(\omega_l=v_l\prod_{m = 1}^{l - 1}(1 - v_m)\), \(v_1,\ldots,v_{H - 1}\) are independent \(\text{Beta}(1,\alpha)\) random variables, and \(v_H = 1\).
- **Prior of the Gaussian factor model**:
\[
(\lambda_{jh}|\pi_h)\sim(1 - \pi_h)N(0,\theta_0)+\pi_hN(0,\theta_{\infty}),\quad(j = 1,\ldots,p;h = 1,\ldots,H)
\]
- **Variational lower bound (ELBO)**:
\[
\text{ELBO}(q)=\log p(y)-\text{KL}(q(\lambda,\eta,\sigma,z,v)\|p(\lambda,\eta,\sigma,z,v|y))
\]
\[
=E_q[\log p(y,\lambda,\eta,\sigma,z,v)]-E_q[\log q(\lambda,\eta,\sigma,z,v)]
\]
Through these methods and formulas, the paper successfully achieves efficient variational Bayesian inference in the Gaussian factor model, providing a new tool for processing large - scale datasets.