Abstract:Abstract To solve key biomedical problems, experimentalists now routinely measure millions or billions of features (dimensions) per sample, with the hope that data science techniques will be able to build accurate data-driven inferences. Because sample sizes are typically orders of magnitude smaller than the dimensionality of these data, valid inferences require finding a low-dimensional representation that preserves the discriminating information (e.g., whether the individual suffers from a particular disease). There is a lack of interpretable supervised dimensionality reduction methods that scale to millions of dimensions with strong statistical theoretical guarantees. We introduce an approach to extending principal components analysis by incorporating class-conditional moment estimates into the low-dimensional projection. The simplest version, Linear Optimal Low-rank projection, incorporates the class-conditional means. We prove, and substantiate with both synthetic and real data benchmarks, that Linear Optimal Low-Rank Projection and its generalizations lead to improved data representations for subsequent classification, while maintaining computational efficiency and scalability. Using multiple brain imaging datasets consisting of more than 150 million features, and several genomics datasets with more than 500,000 features, Linear Optimal Low-Rank Projection outperforms other scalable linear dimensionality reduction techniques in terms of accuracy, while only requiring a few minutes on a standard desktop computer.

Supervised Dimensionality Reduction Via Sequential Semidefinite Programming

Feature Extraction Using Sequential Semidefinite Programming

A variant of the trace quotient formulation for dimensionality reduction

A Unified Framework for Semi-Supervised Dimensionality Reduction

Recursive Dimension Reduction for Semisupervised Learning

Dimension Reduction with Semi-supervised Pairwise Covariance-Preserving Projection

A Perception-Driven Approach to Supervised Dimensionality Reduction for Visualization

Large-margin Weakly Supervised Dimensionality Reduction.

Supervised dimensionality reduction for big data

Prescriptive PCA: Dimensionality Reduction for Two-stage Stochastic Optimization

Deep Dimension Reduction for Supervised Representation Learning

Improving the Performance of MIQP Solvers for Quadratic Programs with Cardinality and Minimum Threshold Constraints: A Semidefinite Program Approach.

Supervised Dimensionality Reduction Methods via Recursive Regression

Deep Nonlinear Sufficient Dimension Reduction

Local Feature Discriminant Projection

Stable Local Dimensionality Reduction Approaches

A Scalable Dual Approach to Semidefinite Metric Learning

Simultaneous Best Subset Selection and Dimension Reduction via Primal-Dual Iterations

Supervised Dimensionality Reduction Technology of Generalized Discriminant Component Analysis and Its Kernelization Forms

A DECOMPOSITION AUGMENTED LAGRANGIAN METHOD FOR LOW-RANK SEMIDEFINITE PROGRAMMING

Decomposed Structured Subsets for Semidefinite Optimization