Abstract:We study a general factor analysis framework where the $n$-by-$p$ data matrix is assumed to follow a general exponential family distribution entry-wise. While this model framework has been proposed before, we here further relax its distributional assumption by using a quasi-likelihood setup. By parameterizing the mean-variance relationship on data entries, we additionally introduce a dispersion parameter and entry-wise weights to model large variations and missing values. The resulting model is thus not only robust to distribution misspecification but also more flexible and able to capture non-Gaussian covariance structures of the data matrix. Our main focus is on efficient computational approaches to perform the factor analysis. Previous modeling frameworks rely on simulated maximum likelihood (SML) to find the factorization solution, but this method was shown to lead to asymptotic bias when the simulated sample size grows slower than the square root of the sample size $n$, eliminating its practical application for data matrices with large $n$. Borrowing from expectation-maximization (EM) and stochastic gradient descent (SGD), we investigate three estimation procedures based on iterative factorization updates. Our proposed solution does not show asymptotic biases, and scales even better for large matrix factorizations with error $O(1/p)$. To support our findings, we conduct simulation experiments and discuss its application in three case studies.

Free Exponential Families as Kernel Families

One-sided Cauchy-Stieltjes Kernel Families

On Cauchy-Stieltjes Kernel Families

On a method to construct exponential families by representation theory

Exponential Iteration and Borel Sets

Stability of Cauchy–Stieltjes Kernel Families by Free and Boolean Convolutions Product

Statistical exponential families: A digest with flash cards

On-line Evolutionary Exponential Family Mixture

Cauchy-Stieltjes kernels families and free multiplicative convolution

Posteriors, conjugacy, and exponential families for completely random measures

Estimation in Functional Regression for General Exponential Families

A Characterization of Conjugate Priors in Exponential Families with Application to Inverse Regression

On $φ$-families of probability distributions

Kernels of discrete convolutions and subdivision operators

A reproducing kernel approach to Lebesgue decomposition

Poisson Processes in Free Probability

Cauchy-Stieltjes Kernel families: Some properties

Free Stein kernels, free moment maps, and higher order derivatives

Kernel-based Approximate Bayesian Inference for Exponential Family Random Graph Models

Computational Approaches for Exponential-Family Factor Analysis