Abstract:We consider a novel Bayesian approach to estimation, uncertainty quantification, and variable selection for a high-dimensional linear regression model under sparsity. The number of predictors can be nearly exponentially large relative to the sample size. We put a conjugate normal prior initially disregarding sparsity, but for making an inference, instead of the original multivariate normal posterior, we use the posterior distribution induced by a map transforming the vector of regression coefficients to a sparse vector obtained by minimizing the sum of squares of deviations plus a suitably scaled $\ell_1$-penalty on the vector. We show that the resulting sparse projection-posterior distribution contracts around the true value of the parameter at the optimal rate adapted to the sparsity of the vector. We show that the true sparsity structure gets a large sparse projection-posterior probability. We further show that an appropriately recentred credible ball has the correct asymptotic frequentist coverage. Finally, we describe how the computational burden can be distributed to many machines, each dealing with only a small fraction of the whole dataset. We conduct a comprehensive simulation study under a variety of settings and found that the proposed method performs well for finite sample sizes. We also apply the method to several real datasets, including the ADNI data, and compare its performance with the state-of-the-art methods. We implemented the method in the \texttt{R} package called \texttt{sparseProj}, and all computations have been carried out using this package.

Uncertainty Quantification in Bayesian Reduced-Rank Sparse Regressions

Bayesian Uncertainty Quantification for Low-Rank Matrix Completion

Bayesian sparse multiple regression for simultaneous rank reduction and variable selection

Estimation of high-dimensional low-rank matrices

Robust reduced-rank regression

Uncertainty Quantification for Sparse Deep Learning

A randomized algorithm to solve reduced rank operator regression

Unified Low-Rank Matrix Estimate Via Penalized Matrix Least Squares Approximation

High-Dimensional Uncertainty Quantification via Active and Rank-Adaptive Tensor Regression

Quantifying Registration Uncertainty With Sparse Bayesian Modelling

Sparse Reduced Rank Huber Regression in High Dimensions

Adaptive estimation of the rank of the coefficient matrix in high dimensional multivariate response regression models

Bayesian High-dimensional Linear Regression with Sparse Projection-posterior

Sparse Bayesian Learning for Ranking.

Multivariate Spatiotemporal Models with Low Rank Coefficient Matrix

High-Dimensional Uncertainty Quantification via Tensor Regression with Rank Determination and Adaptive Sampling

Multivariate reduced rank regression by signal subspace matching

Uncertainty Quantification Metrics for Deep Regression

Weighted Reduced Rank Estimators Under Cointegration Rank Uncertainty

Bayesian singular value regularization via a cumulative shrinkage process

Bayesian Inference for Adaptive Low Rank and Sparse Matrix Estimation