Abstract:Principal component analysis (PCA) is a well-established tool in machine learning and data processing. The principal axes in PCA were shown to be equivalent to the maximum marginal likelihood estimator of the factor loading matrix in a latent factor model for the observed data, assuming that the latent factors are independently distributed as standard normal distributions. However, the independence assumption may be unrealistic for many scenarios such as modeling multiple time series, spatial processes, and functional data, where the outcomes are correlated. In this paper, we introduce the generalized probabilistic principal component analysis (GPPCA) to study the latent factor model for multiple correlated outcomes, where each factor is modeled by a Gaussian process. Our method generalizes the previous probabilistic formulation of PCA (PPCA) by providing the closed-form maximum marginal likelihood estimator of the factor loadings and other parameters. Based on the explicit expression of the precision matrix in the marginal likelihood that we derived, the number of the computational operations is linear to the number of output variables. Furthermore, we also provide the closed-form expression of the marginal likelihood when other covariates are included in the mean structure. We highlight the advantage of GPPCA in terms of the practical relevance, estimation accuracy and computational convenience. Numerical studies of simulated and real data confirm the excellent finite-sample performance of the proposed approach.

Probabilistic Relational PCA.

Sparse Probabilistic Relational Projection.

Robust Principal Component Analysis Based on Maximum Correntropy Criterion

Sparse Probabilistic Principal Component Analysis Model for Plant-Wide Process Monitoring

Privacy Preserving PCA for Multiparty Modeling

Self-Paced Probabilistic Principal Component Analysis for Data with Outliers

Bilinear Probabilistic Principal Component Analysis.

Generalized probabilistic principal component analysis of correlated data

Dynamic Process Monitoring Based on Probabilistic Principle Component Regression

Torus Probabilistic Principal Component Analysis

Probabilistic process monitoring with Bayesian regularization

A Method for Implementing a Probabilistic Model as a Relational Database

MML Probabilistic Principal Component Analysis

Probabilistic PCA for T Distributions.

Improved Algorithms for High-Dimensional Robust Pca

HePPCAT: Probabilistic PCA for Data with Heteroscedastic Noise

Probabilistic Analysis of Kernel Principal Components

Uncertainty-Aware Principal Component Analysis

Robust Bilinear Probabilistic PCA Using a Matrix Variate T Distribution.

Probabilistic Ranking over Relations

On the asymptotic properties of product-PCA under the high-dimensional setting