Structured prior distributions for the covariance matrix in latent factor models

Sarah Elizabeth Heaps,Ian Hyla Jermyn

DOI: https://doi.org/10.1007/s11222-024-10454-0

IF: 2.3241

2024-06-27

Statistics and Computing

Abstract:Factor models are widely used for dimension reduction in the analysis of multivariate data. This is achieved through decomposition of a covariance matrix into the sum of two components. Through a latent factor representation, they can be interpreted as a diagonal matrix of idiosyncratic variances and a shared variation matrix, that is, the product of a factor loadings matrix and its transpose. If , this defines a parsimonious factorisation of the covariance matrix. Historically, little attention has been paid to incorporating prior information in Bayesian analyses using factor models where, at best, the prior for the factor loadings is order invariant. In this work, a class of structured priors is developed that can encode ideas of dependence structure about the shared variation matrix. The construction allows data-informed shrinkage towards sensible parametric structures while also facilitating inference over the number of factors. Using an unconstrained reparameterisation of stationary vector autoregressions, the methodology is extended to stationary dynamic factor models. For computational inference, parameter-expanded Markov chain Monte Carlo samplers are proposed, including an efficient adaptive Gibbs sampler. Two substantive applications showcase the scope of the methodology and its inferential benefits.

statistics & probability,computer science, theory & methods

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is how to effectively combine prior information in the Bayesian factor model, especially the prior distribution encoding the dependence structure in the shared variation matrix. Specifically, the authors propose a class of structured prior distributions. These prior distributions can reflect prior beliefs about the dependence relationships among variables, while allowing data - driven shrinkage towards reasonable parameter structures and supporting the inference of the number of factors. In addition, by introducing an unconstrained re - parameterization method, this study also extends this method to the stationary dynamic factor model. This is the first general dynamic factor model prior and its associated inference scheme that restricts the inference within the stationary region without imposing additional restrictions. The paper also proposes efficient parameter - expanded Markov chain Monte Carlo (MCMC) samplers, including an efficient adaptive Gibbs sampler for computational inference.

The key contribution of the paper lies in providing a flexible framework that can incorporate initial beliefs about the shared variation matrix into the prior of the factor loading matrix. By exploiting the algebraic relationship between the two, this method is more flexible than other methods described in the literature. This not only improves the interpretability of the model but also enhances the effectiveness of dimensionality reduction analysis for high - dimensional data.