Abstract:Principal component analysis (PCA) is a well-established tool in machine learning and data processing. The principal axes in PCA were shown to be equivalent to the maximum marginal likelihood estimator of the factor loading matrix in a latent factor model for the observed data, assuming that the latent factors are independently distributed as standard normal distributions. However, the independence assumption may be unrealistic for many scenarios such as modeling multiple time series, spatial processes, and functional data, where the outcomes are correlated. In this paper, we introduce the generalized probabilistic principal component analysis (GPPCA) to study the latent factor model for multiple correlated outcomes, where each factor is modeled by a Gaussian process. Our method generalizes the previous probabilistic formulation of PCA (PPCA) by providing the closed-form maximum marginal likelihood estimator of the factor loadings and other parameters. Based on the explicit expression of the precision matrix in the marginal likelihood that we derived, the number of the computational operations is linear to the number of output variables. Furthermore, we also provide the closed-form expression of the marginal likelihood when other covariates are included in the mean structure. We highlight the advantage of GPPCA in terms of the practical relevance, estimation accuracy and computational convenience. Numerical studies of simulated and real data confirm the excellent finite-sample performance of the proposed approach.

Probabilistic PCA for T Distributions.

Robust Bilinear Probabilistic PCA Using a Matrix Variate T Distribution.

Robust Principal Component Analysis Based on Maximum Correntropy Criterion

Torus Probabilistic Principal Component Analysis

Robust modeling of mixture probabilistic principal component analysis and process monitoring application

Uncertainty-Aware Principal Component Analysis

Robust factored principal component analysis for matrix-valued outlier accommodation and detection

Self-Paced Probabilistic Principal Component Analysis for Data with Outliers

Generalized probabilistic principal component analysis of correlated data

MML Probabilistic Principal Component Analysis

Robust Covariance Estimation for Distributed Principal Component Analysis

Distributed Tensor Principal Component Analysis

Bilinear Probabilistic Principal Component Analysis.

Probabilistic Relational PCA.

Latent Variable Models for Dimensionality Reduction

Two derivations of Principal Component Analysis on datasets of distributions

Robust PCA for High Dimensional Data based on Characteristic Transformation

Mixture Probabilistic Principal Geodesic Analysis

Tucker tensor factor models: matricization and mode-wise PCA estimation

Matrix normal PCA for interpretable dimension reduction and graphical noise modeling

Dynamic Principal Component Analysis in High Dimensions