Abstract:The issue of honesty in constructing confidence sets arises in nonparametric regression. While optimal rate in nonparametric estimation can be achieved and utilized to construct sharp confidence sets, severe degradation of confidence level often happens after estimating the degree of smoothness. Similarly, for high-dimensional regression, oracle inequalities for sparse estimators could be utilized to construct sharp confidence sets. Yet, the degree of sparsity itself is unknown and needs to be estimated, which causes the honesty problem. To resolve this issue, we develop a novel method to construct honest confidence sets for sparse high-dimensional linear regression. The key idea in our construction is to separate signals into a strong and a weak group, and then construct confidence sets for each group separately. This is achieved by a projection and shrinkage approach, the latter implemented via Stein estimation and the associated Stein unbiased risk estimate. Our confidence set is honest over the full parameter space without any sparsity constraints, while its size adapts to the optimal rate of n−1/4 when the true parameter is indeed sparse. Moreover, under some form of a separation assumption between the strong and weak signals, the diameter of our confidence set can achieve a faster rate than existing methods. Through extensive numerical comparisons on both simulated and real data, we demonstrate that our method outperforms other competitors with big margins for finite samples, including oracle methods built upon the true sparsity of the underlying model.

James–Stein estimation of the first principal component

Bias Reduction in Instrumental Variable Estimation through First-Stage Shrinkage

An Empirical Bayes Approach to Shrinkage Estimation on the Manifold of Symmetric Positive-Definite Matrices

Augmented Estimation of Principal Component Subspace in High Dimensions

Estimation of a high-dimensional covariance matrix with the Stein loss

Nonparametric Stein-type shrinkage covariance matrix estimators in high-dimensional settings

EIGENVALUE DISTRIBUTIONS OF VARIANCE COMPONENTS ESTIMATORS IN HIGH-DIMENSIONAL RANDOM EFFECTS MODELS

James-Stein Estimation in Quantum Gaussian Sensing

High-dimensional Index Volatility Models via Stein's Identity

Improving Normalization with the James-Stein Estimator

Approximation bounds for sparse principal component analysis

The High-Dimensional Asymptotics of Principal Component Regression

Dynamic Principal Component Analysis in High Dimensions

Principal Component Pursuit with Reduced Linear Measurements

Principal component analysis balancing prediction and approximation accuracy for spatial data

Honest Confidence Sets for High-Dimensional Regression by Projection and Shrinkage

Sparse principal component analysis by choice of norm

On the principal components of sample covariance matrices

Covariance Estimation in High Dimensions Via Kronecker Product Expansions

When and why are principal component scores a good tool for visualizing high-dimensional data?

A note on the variance in principal component regression