Abstract:Principal component analysis (PCA) is a well-established tool in machine learning and data processing. The principal axes in PCA were shown to be equivalent to the maximum marginal likelihood estimator of the factor loading matrix in a latent factor model for the observed data, assuming that the latent factors are independently distributed as standard normal distributions. However, the independence assumption may be unrealistic for many scenarios such as modeling multiple time series, spatial processes, and functional data, where the outcomes are correlated. In this paper, we introduce the generalized probabilistic principal component analysis (GPPCA) to study the latent factor model for multiple correlated outcomes, where each factor is modeled by a Gaussian process. Our method generalizes the previous probabilistic formulation of PCA (PPCA) by providing the closed-form maximum marginal likelihood estimator of the factor loadings and other parameters. Based on the explicit expression of the precision matrix in the marginal likelihood that we derived, the number of the computational operations is linear to the number of output variables. Furthermore, we also provide the closed-form expression of the marginal likelihood when other covariates are included in the mean structure. We highlight the advantage of GPPCA in terms of the practical relevance, estimation accuracy and computational convenience. Numerical studies of simulated and real data confirm the excellent finite-sample performance of the proposed approach.

A Dual Formulation for Probabilistic Principal Component Analysis

Robust Principal Component Analysis Based on Maximum Correntropy Criterion

Probabilistic Analysis of Kernel Principal Components

Bilinear Probabilistic Principal Component Analysis.

Dual Principal Component Pursuit: Probability Analysis and Efficient Algorithms

Best Pair Formulation & Accelerated Scheme for Non-convex Principal Component Pursuit.

Uncertainty-Aware Principal Component Analysis

Mixture of Bilateral-Projection Two-Dimensional Probabilistic Principal Component Analysis

An Infinite Dimensional Analysis of Kernel Principal Components

Dynamic Principal Component Analysis in High Dimensions

Probabilistic PCA for T Distributions.

Two derivations of Principal Component Analysis on datasets of distributions

Dynamic Principal Subspaces in High Dimensions

Dynamic Principal Subspaces with Sparsity in High Dimensions

MML Probabilistic Principal Component Analysis

Feature Extraction Based on Mixture Probabilistic Kernel Principal Component Analysis

Generalized probabilistic principal component analysis of correlated data

Latent Variable Models for Dimensionality Reduction

Positive Definite Kernels, Algorithms, Frames, and Approximations

A Kernelizable Primal-Dual Formulation of the Multilinear Singular Value Decomposition

Generalisation bounds for kernel PCA through PAC‐Bayes learning