Abstract:Principal component analysis (PCA) is one of the most fundamental tools in machine learning with broad use as a dimensionality reduction and denoising tool. In the later setting, while PCA is known to be effective at subspace recovery and is proven to aid clustering algorithms in some specific settings, its improvement of noisy data is still not well quantified in general. In this paper, we propose a novel metric called \emph{compression ratio} to capture the effect of PCA on high-dimensional noisy data. We show that, for data with \emph{underlying community structure}, PCA significantly reduces the distance of data points belonging to the same community while reducing inter-community distance relatively mildly. We explain this phenomenon through both theoretical proofs and experiments on real-world data. Building on this new metric, we design a straightforward algorithm that could be used to detect outliers. Roughly speaking, we argue that points that have a \emph{lower variance of compression ratio} do not share a \emph{common signal} with others (hence could be considered outliers). We provide theoretical justification for this simple outlier detection algorithm and use simulations to demonstrate that our method is competitive with popular outlier detection tools. Finally, we run experiments on real-world high-dimension noisy data (single-cell RNA-seq) to show that removing points from these datasets via our outlier detection method improves the accuracy of clustering algorithms. Our method is very competitive with popular outlier detection tools in this task.

Double-Attentive Principle Component Analysis

Robust Principal Component Analysis Based on Maximum Correntropy Criterion

Criteria for 2DPCA Superior to PCA in Image Feature Extraction

Low-light Level Image De-Noising Algorithm Based on PCA

A Pca Based Automatic Image Categorization Approach Using Dominant Color Features

Double L<sub>2,p</sub>-norm based PCA for feature extraction

Arbitrary Triangle Structure Adaptive Mean PCA and Image Recognition

Dual Principal Component Pursuit: Probability Analysis and Efficient Algorithms

Robust Two-Dimensional Principal Component Analysis Via Alternating Optimization

A Robust Optimal Mean Cosine Angle 2DPCA for Image Feature Extraction

Bilateral Two-Dimensional Principal Component Analysis with its Application to Face Recognition

Discriminative Principal Component Analysis: A REVERSE THINKING

Two-Dimensional Whitening Reconstruction for Enhancing Robustness of Principal Component Analysis

Robust Principal Component Analysis via Discriminant Sample Weight Learning

Robust 2D Principal Component Analysis: A Structured Sparsity Regularized Approach.

Robust Principal Component Analysis via Joint l(2,1)-Norms Minimization

The Equivalence of Two-Dimensional PCA to Line-Based PCA

Capturing the Denoising Effect of PCA via Compression Ratio

Normalized Robust PCA With Adaptive Reconstruction Error Minimization

Robust Principal Component Analysis Via Joint ℓ<inf>2,1</inf>-Norms Minimization

( 2 D ) 2 PCA : 2-Directional 2-Dimensional PCA for Efficient Face Representation and Recognition