Matrix normal PCA for interpretable dimension reduction and graphical noise modeling

Chihao Zhang,Kuo Gai,Shihua Zhang
DOI: https://doi.org/10.1016/j.patcog.2024.110591
IF: 8
2024-05-16
Pattern Recognition
Abstract:Principal component analysis (PCA) is one of the most widely used dimension reduction and multivariate statistical techniques. From a probabilistic perspective, PCA seeks a low-dimensional representation of data in the presence of independent identical Gaussian noise. Probabilistic PCA (PPCA) and its variants have been extensively studied for decades. Most of them assume the underlying noise follows a certain independent identical distribution. However, the noise in the real world is usually complicated and structured. To address this challenge, some variants of PCA for non-IID data have been proposed. However, most of the existing methods only assume that the noise is correlated in the feature space while there may exist two-way structured noise. To this end, we propose a powerful and intuitive PCA method (MN-PCA) through modeling the graphical noise by the matrix normal distribution, which enables us to explore the structure of noise in both the feature space and the sample space. MN-PCA obtains a low-rank representation of data and the structure of noise simultaneously. And it can be explained as approximating data over the generalized Mahalanobis distance. We first solve this model by a standard approach—maximizing the regularized likelihood —and then develop a novel algorithm that exploits the Wasserstein distance, which is more robust. Extensive experiments on various data demonstrate their effectiveness.
computer science, artificial intelligence,engineering, electrical & electronic
What problem does this paper attempt to address?