Abstract:Rotational post hoc transformations have traditionally played a key role in enhancing the interpretability of factor analysis. Regularization methods also serve to achieve this goal by prioritizing sparse loading matrices. In this work, we bridge these two paradigms with a unifying Bayesian framework. Our approach deploys intermediate factor rotations throughout the learning process, greatly enhancing the effectiveness of sparsity inducing priors. These automatic rotations to sparsity are embedded within a PXL-EM algorithm, a Bayesian variant of parameter-expanded EM for posterior mode detection. By iterating between soft-thresholding of small factor loadings and transformations of the factor basis, we obtain (a) dramatic accelerations, (b) robustness against poor initializations, and (c) better oriented sparse solutions. To avoid the prespecification of the factor cardinality, we extend the loading matrix to have infinitely many columns with the Indian buffet process (IBP) prior. The factor dimensionality is learned from the posterior, which is shown to concentrate on sparse matrices. Our deployment of PXL-EM performs a dynamic posterior exploration, outputting a solution path indexed by a sequence of spike-and-slab priors. For accurate recovery of the factor loadings, we deploy the spike-and-slab LASSO prior, a two-component refinement of the Laplace prior. A companion criterion, motivated as an integral lower bound, is provided to effectively select the best recovery. The potential of the proposed procedure is demonstrated on both simulated and real high-dimensional data, which would render posterior simulation impractical. Supplementary materials for this article are available online.

Geometric vector orthogonal rotation method in multiple-factor analysis

Iterative Algorithm for Identifying Multi-Modes Using Real Orthogonal Polynomials

Local minima and factor rotations in exploratory factor analysis.

Orthogonal inter-battery factor analysis

Varimax rotation based on gradient projection needs between 10 and more than 500 random start loading matrices for optimal performance

A New Algorithm for Computing Disjoint Orthogonal Components in the Parallel Factor Analysis Model with Simulations and Applications to Real-World Data

Analyzing Vector Orthogonalization Algorithms

Fast Bayesian Factor Analysis via Automatic Rotations to Sparsity

Optimal vintage factor analysis with deflation varimax

Robust oblique Target-rotation for small samples

Enhancing Interpretability in Factor Analysis by Means of Mathematical Optimization

Rotation Local Solutions in Multidimensional Item Response Theory Models

3d Orthographic Reconstruction Based On Robust Factorization Method With Outliers

Identifying Observed Factors in Approximate Factor Models: Estimation and Hypothesis Testing

EIGENVALUE DISTRIBUTIONS OF VARIANCE COMPONENTS ESTIMATORS IN HIGH-DIMENSIONAL RANDOM EFFECTS MODELS

Orthogonal multi-view analysis by successive approximations via eigenvectors

Orthogonal projection analysis

A two-step basis vector extraction strategy for multiset variable correlation analysis

Bayesian Extensive-Rank Matrix Factorization with Rotational Invariant Priors

A Novel Rotating Orthogonal Method for Combinatorial Optimization