Abstract:High-dimensional data are commonly seen in modern statistical applications, variable selection methods play indispensable roles in identifying the critical features for scientific discoveries. Traditional best subset selection methods are computationally intractable with a large number of features, while regularization methods such as Lasso, SCAD and their variants perform poorly in ultrahigh-dimensional data due to low computational efficiency and unstable algorithm. Sure screening methods have become popular alternatives by first rapidly reducing the dimension using simple measures such as marginal correlation then applying any regularization methods. A number of screening methods for different models or problems have been developed, however, none of the methods have targeted at data with heavy tailedness, which is another important characteristics of modern big data. In this paper, we propose a robust distance correlation (``RDC'') based sure screening method to perform screening in ultrahigh-dimensional regression with heavy-tailed data. The proposed method shares the same good properties as the original model-free distance correlation based screening while has additional merit of robustly estimating the distance correlation when data is heavy-tailed and improves the model selection performance in screening. We conducted extensive simulations under different scenarios of heavy tailedness to demonstrate the advantage of our proposed procedure as compared to other existing model-based or model-free screening procedures with improved feature selection and prediction performance. We also applied the method to high-dimensional heavy-tailed RNA sequencing (RNA-seq) data of The Cancer Genome Atlas (TCGA) pancreatic cancer cohort and RDC was shown to outperform the other methods in prioritizing the most essential and biologically meaningful genes.

An Adaptive Resampling Test for Detecting the Presence of Significant Predictors

Bayesian Updating and Sequential Testing: Overcoming Inferential Limitations of Screening Tests

Empirical Likelihood Based Tests for Detecting the Presence of Significant Predictors in Marginal Quantile Regression

EFFICIENT ESTIMATION OF THE MAXIMAL ASSOCIATION BETWEEN MULTIPLE PREDICTORS AND A SURVIVAL OUTCOME

A Non-Parametric Method for Building Predictive Genetic Tests on High-Dimensional Data

Threshold Selection in Feature Screening for Error Rate Control*

Conditional screening for ultra-high dimensional covariates with survival outcomes

Unbiased Prediction and Feature Selection in High-Dimensional Survival Regression

Sure screening for estimating equations in ultra-high dimensions

Shrinkage-based regularization tests for high-dimensional data with application to gene set analysis

Principal Components Adjusted Variable Screening

Screen then select: a strategy for correlated predictors in high-dimensional quantile regression

A note on marginal correlation based screening

An Improved Variable Selection Procedure for Adaptive Lasso in High-Dimensional Survival Analysis.

Efficient screening of predictive biomarkers for individual treatment selection

Nonparametric screening for additive quantile regression in ultra-high dimension

An Adaptive Test on High-dimensional Parameters in Generalized Linear Models

Statistical Methods for Selective Biomarker Testing

Model-free screening procedure for ultrahigh-dimensional survival data based on Hilbert-Schmidt independence criterion

Robust distance correlation for variable screening

Adverse Subpopulation Regression for Multivariate Outcomes with High-Dimensional Predictors