Abstract:Rotational post hoc transformations have traditionally played a key role in enhancing the interpretability of factor analysis. Regularization methods also serve to achieve this goal by prioritizing sparse loading matrices. In this work, we bridge these two paradigms with a unifying Bayesian framework. Our approach deploys intermediate factor rotations throughout the learning process, greatly enhancing the effectiveness of sparsity inducing priors. These automatic rotations to sparsity are embedded within a PXL-EM algorithm, a Bayesian variant of parameter-expanded EM for posterior mode detection. By iterating between soft-thresholding of small factor loadings and transformations of the factor basis, we obtain (a) dramatic accelerations, (b) robustness against poor initializations, and (c) better oriented sparse solutions. To avoid the prespecification of the factor cardinality, we extend the loading matrix to have infinitely many columns with the Indian buffet process (IBP) prior. The factor dimensionality is learned from the posterior, which is shown to concentrate on sparse matrices. Our deployment of PXL-EM performs a dynamic posterior exploration, outputting a solution path indexed by a sequence of spike-and-slab priors. For accurate recovery of the factor loadings, we deploy the spike-and-slab LASSO prior, a two-component refinement of the Laplace prior. A companion criterion, motivated as an integral lower bound, is provided to effectively select the best recovery. The potential of the proposed procedure is demonstrated on both simulated and real high-dimensional data, which would render posterior simulation impractical. Supplementary materials for this article are available online.

Sparse Bayesian factor analysis when the number of factors is unknown

Bayesian factor-adjusted sparse regression

Sparse factor models of high dimension

Inference algorithms and learning theory for Bayesian sparse factor analysis

Fast Bayesian Factor Analysis via Automatic Rotations to Sparsity

A Comparison of Bayesian Inference Techniques for Sparse Factor Analysis

Dynamic Sparse Factor Analysis

Generalized cumulative shrinkage process priors with applications to sparse Bayesian factor analysis

Incorporating graph information in Bayesian factor analysis with robust and adaptive shrinkage priors

Disjunct Support Spike and Slab Priors for Variable Selection in Regression under Quasi-sparseness

Exponential Family Factors for Bayesian Factor Analysis

Efficient Probabilistic Latent Semantic Analysis with Sparsity Control

Algebraic Sparse Factor Analysis

Bayesian factor selection in a hybrid approach to confirmatory factor analysis

Structured prior distributions for the covariance matrix in latent factor models

Enhancing Scalability in Bayesian Nonparametric Factor Analysis of Spatiotemporal Data

Fast Laplace Approximation for Sparse Bayesian Spike and Slab Models

Interpretable Sparse Proximate Factors for Large Dimensions

A sparse factor model for clustering high‐dimensional longitudinal data

Bayesian High-dimensional Linear Regression with Sparse Projection-posterior

The sparse dynamic factor model: a regularised quasi-maximum likelihood approach