Abstract:Dimensionality reduction (DR) is frequently applied during the analysis of high-dimensional data. Both a means of denoising and simplification, it can be beneficial for the majority of modern biological datasets, in which it's not uncommon to have hundreds or even millions of simultaneous measurements collected for a single sample. Because of "the curse of dimensionality," many statistical methods lack power when applied to high-dimensional data. Even if the number of collected data points is large, they remain sparsely submerged in a voluminous high-dimensional space that is practically impossible to explore exhaustively (see chapter 12 [<a class="ref-tip" href="#pcbi.1006907.ref001">1</a>]). By reducing the dimensionality of the data, you can often alleviate this challenging and troublesome phenomenon. Low-dimensional data representations that remove noise but retain the signal of interest can be instrumental in understanding hidden structures and patterns. Original high-dimensional data often contain measurements on uninformative or redundant variables. DR can be viewed as a method for latent feature extraction. It is also frequently used for data compression, exploration, and visualization. Although many DR techniques have been developed and implemented in standard data analytic pipelines, they are easy to misuse, and their results are often misinterpreted in practice. This article presents a set of useful guidelines for practitioners specifying how to correctly perform DR, interpret its output, and communicate results. Note that this is not a review article, and we recommend some important reviews in the references.

System and method for incrementally converting high dimensional data into low dimensional data

System and method for converting collected high-dimension data into low-dimension data

Method, apparatus and system for processing network data

Data Compression and Storage under High Speed Network

Data-driven Dimension Reduction for High-Dimensional Random Vibration Systems with Cubic Nonlinearity

Deep Clustering and Visualization for End-to-End High-Dimensional Data Analysis.

A Dimensional Reduction Guiding Deep Learning Architecture for 3D Shape Retrieval

Feature dimensionality reduction: a review

Enhanced Locality Sensitive Clustering in High Dimensional Space

Fusion of effective dimension reduction and discriminative dictionary learning for high-dimensional classification

Secure and verifiable outsourced data dimension reduction on dynamic data

Framelet pooling aided deep learning network: the method to process high dimensional medical data

Dimension Reduction Algorithm Based on Adaptive Maximum Linear Neighborhood Selection in Edge Computing

Interpreting High-Dimensional Projections with Capacity.

Accelerated Variance Reduced Block Coordinate Descent

<Bold>An Implementation Framework For Kernel Methods With High-Dimensional Patterns</Bold>

Distributed Sufficient Dimension Reduction for Heterogeneous Massive Data

Supervised dimensionality reduction for big data

An Incremental Dimensionality Reduction Method for Visualizing Streaming Multidimensional Data

Ten quick tips for effective dimensionality reduction