Abstract:Dimensionality reduction (DR) algorithms are diverse and widely used for analyzing high-dimensional data. Various metrics and tools have been proposed to evaluate and interpret the DR results. However, most metrics and methods fail to be well generalized to measure any DR results from the perspective of original distribution fidelity or lack interactive exploration of DR results. There is still a need for more intuitive and quantitative analysis to interactively explore high-dimensional data and improve interpretability. We propose a metric and a generalized algorithm-agnostic approach based on the concept of capacity to evaluate and analyze the DR results. Based on our approach, we develop a visual analytic system HiLow for exploring high-dimensional data and projections. We also propose a mixed-initiative recommendation algorithm that assists users in interactively DR results manipulation. Users can compare the differences in data distribution after the interaction through HiLow. Furthermore, we propose a novel visualization design focusing on quantitative analysis of differences between high and low-dimensional data distributions. Finally, through user study and case studies, we validate the effectiveness of our approach and system in enhancing the interpretability of projections and analyzing the distribution of high and low-dimensional data.

Direction-Projection-Permutation for High Dimensional Hypothesis Tests

diproperm: An R Package for the DiProPerm Test

A More Powerful Two-Sample Test in High Dimensions using Random Projection

Penalized Independence Rule for Testing High-Dimensional Hypotheses

Interpreting High-Dimensional Projections with Capacity.

Residual Permutation Test for High-Dimensional Regression Coefficient Testing

A PERMUTATION TEST FOR TWO-SAMPLE MEANS AND SIGNAL IDENTIFICATION OF HIGH-DIMENSIONAL DATA

Linear Hypothesis Testing in Linear Models With High-Dimensional Responses

Projection Test for Mean Vector in High Dimensions

Nonparametric High-Dimensional Multi-Sample Tests based on Graph Theory

A Unified Framework for Testing High Dimensional Parameters: A Data-Adaptive Approach.

Randomized tests for high-dimensional regression: A more efficient and powerful solution

A Unified Framework for Testing High Dimensional Parameters: A Data-Adaptive Approach

Projection Tests for Regression Coefficients in High-Dimensional Partial Linear Models

Simulation-Based Hypothesis Testing of High Dimensional Means Under Covariance Heterogeneity

Shrinkage-based regularization tests for high-dimensional data with application to gene set analysis

Model-Free Statistical Inference on High-Dimensional Data

On the Decreasing Power of Kernel and Distance Based Nonparametric Hypothesis Tests in High Dimensions

Simultaneous test for linear model via projection

Statistical divergences in high-dimensional hypothesis testing and a modern technique for estimating them

Testing and Confidence Intervals for High Dimensional Proportional Hazards Model