Abstract:Jianqing Fan, Dong Wang, Kaizheng Wang, Ziwei Zhu. Principal component analysis (PCA) is fundamental to statistical machine learning. It extracts latent principal factors that contribute to the most variation of the data. When data are stored across multiple machines, however, communication cost can prohibit the computation of PCA in a central location and distributed algorithms for PCA are thus needed. This paper proposes and studies a distributed PCA algorithm: each node machine computes the top $K$ eigenvectors and transmits them to the central server; the central server then aggregates the information from all the node machines and conducts a PCA based on the aggregated information. We investigate the bias and variance for the resulting distributed estimator of the top $K$ eigenvectors. In particular, we show that for distributions with symmetric innovation, the empirical top eigenspaces are unbiased, and hence the distributed PCA is "unbiased." We derive the rate of convergence for distributed PCA estimators, which depends explicitly on the effective rank of covariance, eigengap, and the number of machines. We show that when the number of machines is not unreasonably large, the distributed PCA performs as well as the whole sample PCA, even without full access of whole data. The theoretical results are verified by an extensive simulation study. We also extend our analysis to the heterogeneous case where the population covariance matrices are different across local machines but share similar top eigenstructures.

Research on Distributed Heterogeneous Data PCA Algorithm Based on Cloud Platform

Privacy Preserving PCA for Multiparty Modeling

IMPLEMENT DATA SHARING OVER NETWORK HETEROGENEOUS DATABASES BY DATA DIMENSION REDUCTION METHOD

Distributed estimation of principal eigenspaces

A Communication-Efficient Distributed Algorithm for Kernel Principal Component Analysis.

Robust Covariance Estimation for Distributed Principal Component Analysis

Distributed Parallel PCA for Modeling and Monitoring of Large-Scale Plant-Wide Processes with Big Data.

Distributed Estimation for Principal Component Analysis: An Enlarged Eigenspace Analysis

Outsourcing Eigen-Decomposition and Singular Value Decomposition of Large Matrix to a Public Cloud

PPHOPCM Privacy-Preserving High-order Possibilistic C-Means Algorithm for Big Data Clustering with Cloud Computing

HePPCAT: Probabilistic PCA for Data with Heteroscedastic Noise

Streaming Probabilistic PCA for Missing Data with Heteroscedastic Noise

Research on heterogeneous model based key cloud computing technologies

Quantum Ciphertext Dimension Reduction Scheme for Homomorphic Encrypted Data

A linearly convergent algorithm for distributed principal component analysis

HaDPA: A Data-Partition Algorithm for Data Parallel Applications on Heterogeneous HPC Platforms

Cloud-based Privacy- and Integrity-Protecting Density Peaks Clustering.

A Covariance-Free Iterative Algorithm for Distributed Principal Component Analysis on Vertically Partitioned Data.

Scalable and Privacy-Preserving Federated Principal Component Analysis

Online Distributed Estimation of Principal Eigenspaces

Secure and Efficient Outsourcing of PCA-Based Face Recognition