JAAA from a geographic perspective.

J. Jerger

DOI: https://doi.org/10.3766/jaaa.22.9.1

2011-10-01

Abstract:

What problem does this paper attempt to address?

Sparsified simultaneous confidence intervals for high-dimensional linear models

Xiaorui Zhu,Yichen Qin,Peng Wang

DOI: https://doi.org/10.1007/s00184-024-00975-z

IF: 0.96

2024-10-31

Metrika

Abstract:Statistical inference of the high-dimensional regression coefficients is challenging because the uncertainty introduced by the model selection procedure is hard to account for. Currently, the inference of the model and the inference of the coefficients are separately sought. A critical question remains unsettled; that is, is it possible to embed the inference of the model into the simultaneous inference of the coefficients? If so, then how to properly design a simultaneous inference tool with desired properties? To this end, we propose a notion of simultaneous confidence intervals called the sparsified simultaneous confidence intervals (SSCI). Our intervals are sparse in the sense that some of the intervals' upper and lower bounds are shrunken to zero (i.e., [0, 0]), indicating the unimportance of the corresponding covariates. These covariates should be excluded from the final model. The rest of the intervals, either containing zero (e.g., or [0, 1]) or not containing zero (e.g., [2, 3]), indicate the plausible and significant covariates, respectively. The SSCI intuitively suggests a lower-bound model with significant covariates only and an upper-bound model with plausible and significant covariates. The proposed method can be coupled with various selection procedures, making it ideal for comparing their uncertainty. For the proposed method, we establish desirable asymptotic properties, develop intuitive graphical tools for visualization, and justify its superior performance through simulation and real data analysis.

statistics & probability
Efficient Inference on High-Dimensional Linear Models with Missing Outcomes

Yikun Zhang,Alexander Giessing,Yen-Chi Chen

2024-12-10

Abstract:This paper is concerned with inference on the regression function of a high-dimensional linear model when outcomes are missing at random. We propose an estimator which combines a Lasso pilot estimate of the regression function with a bias correction term based on the weighted residuals of the Lasso regression. The weights depend on estimates of the missingness probabilities (propensity scores) and solve a convex optimization program that trades off bias and variance optimally. Provided that the propensity scores can be pointwise consistently estimated at in-sample data points, our proposed estimator for the regression function is asymptotically normal and semi-parametrically efficient among all asymptotically linear estimators. Furthermore, the proposed estimator keeps its asymptotic properties even if the propensity scores are estimated by modern machine learning techniques. We validate the finite-sample performance of the proposed estimator through comparative simulation studies and the real-world problem of inferring the stellar masses of galaxies in the Sloan Digital Sky Survey.

Methodology,Statistics Theory,Applications
Inference for High-Dimensional Linear Expectile Regression with De-Biasing Method

Xiang Li,Yu-Ning Li,Li-Xin Zhang,Jun Zhao

DOI: https://doi.org/10.1016/j.csda.2024.107997

IF: 2.035

2024-01-01

Computational Statistics & Data Analysis

Abstract:The methodology for the inference problem in high-dimensional linear expectile regression is developed. By transforming the expectile loss into a weighted-least-squares form and applying a de-biasing strategy, Wald-type tests for multiple constraints within a regularized framework are established. An estimator for the pseudo-inverse of the generalized Hessian matrix in high dimension is constructed using general amenable regularizers, including Lasso and SCAD, with its consistency demonstrated through a novel proof technique. Simulation studies and real data applications demonstrate the efficacy of the proposed test statistic in both homoscedastic and heteroscedastic scenarios.
Statistical Inference and Large-scale Multiple Testing for High-dimensional Regression Models

T. Tony Cai,Zijian Guo,Yin Xia

DOI: https://doi.org/10.48550/arXiv.2301.10392

2023-01-25

Abstract:This paper presents a selective survey of recent developments in statistical inference and multiple testing for high-dimensional regression models, including linear and logistic regression. We examine the construction of confidence intervals and hypothesis tests for various low-dimensional objectives such as regression coefficients and linear and quadratic functionals. The key technique is to generate debiased and desparsified estimators for the targeted low-dimensional objectives and estimate their uncertainty. In addition to covering the motivations for and intuitions behind these statistical methods, we also discuss their optimality and adaptivity in the context of high-dimensional inference. In addition, we review the recent development of statistical inference based on multiple regression models and the advancement of large-scale multiple testing for high-dimensional regression. The R package SIHR has implemented some of the high-dimensional inference methods discussed in this paper.

Methodology,Statistics Theory
Model-Free Statistical Inference on High-Dimensional Data

Xu Guo,Runze Li,Zhe Zhang,Changliang Zou

DOI: https://doi.org/10.1080/01621459.2024.2310314

IF: 4.369

2024-03-10

Journal of the American Statistical Association

Abstract:This article aims to develop an effective model-free inference procedure for high-dimensional data. We first reformulate the hypothesis testing problem via sufficient dimension reduction framework. With the aid of new reformulation, we propose a new test statistic and show that its asymptotic distribution is χ2 distribution whose degree of freedom does not depend on the unknown population distribution. We further conduct power analysis under local alternative hypotheses. In addition, we study how to control the false discovery rate of the proposed χ2 tests, which are correlated, to identify important predictors under a model-free framework. To this end, we propose a multiple testing procedure and establish its theoretical guarantees. Monte Carlo simulation studies are conducted to assess the performance of the proposed tests and an empirical analysis of a real-world dataset is used to illustrate the proposed methodology. Supplementary materials for this article are available online including a standardized description of the materials available for reproducing the work.

statistics & probability
Inference in High-Dimensional Linear Measurement Error Models

Mengyan Li,Runze Li,Yanyuan Ma

DOI: https://doi.org/10.48550/arXiv.2001.10142

2020-01-28

Abstract:For a high-dimensional linear model with a finite number of covariates measured with error, we study statistical inference on the parameters associated with the error-prone covariates, and propose a new corrected decorrelated score test and the corresponding one-step estimator. We further establish asymptotic properties of the newly proposed test statistic and the one-step estimator. Under local alternatives, we show that the limiting distribution of our corrected decorrelated score test statistic is non-central normal. The finite-sample performance of the proposed inference procedure is examined through simulation studies. We further illustrate the proposed procedure via an empirical analysis of a real data example.

Methodology
Doubly Robust Semiparametric Inference Using Regularized Calibrated Estimation with High-dimensional Data

Satyajit Ghosh,Zhiqiang Tan

DOI: https://doi.org/10.48550/arXiv.2009.12033

2020-09-25

Abstract:Consider semiparametric estimation where a doubly robust estimating function for a low-dimensional parameter is available, depending on two working models. With high-dimensional data, we develop regularized calibrated estimation as a general method for estimating the parameters in the two working models, such that valid Wald confidence intervals can be obtained for the parameter of interest under suitable sparsity conditions if either of the two working models is correctly specified. We propose a computationally tractable two-step algorithm and provide rigorous theoretical analysis which justifies sufficiently fast rates of convergence for the regularized calibrated estimators in spite of sequential construction and establishes a desired asymptotic expansion for the doubly robust estimator. As concrete examples, we discuss applications to partially linear, log-linear, and logistic models and estimation of average treatment effects. Numerical studies in the former three examples demonstrate superior performance of our method, compared with debiased Lasso.

Methodology
Simultaneous Inference for High-Dimensional Linear Models

Xianyang Zhang,Guang Cheng

DOI: https://doi.org/10.1080/01621459.2016.1166114

IF: 4.369

2017-04-03

Journal of the American Statistical Association

Abstract:This article proposes a bootstrap-assisted procedure to conduct simultaneous inference for high-dimensional sparse linear models based on the recent desparsifying Lasso estimator. Our procedure allows the dimension of the parameter vector of interest to be exponentially larger than sample size, and it automatically accounts for the dependence within the desparsifying Lasso estimator. Moreover, our simultaneous testing method can be naturally coupled with the margin screening to enhance its power in sparse testing with a reduced computational cost, or with the step-down method to provide a strong control for the family-wise error rate. In theory, we prove that our simultaneous testing procedure asymptotically achieves the prespecified significance level, and enjoys certain optimality in terms of its power even when the model errors are non-Gaussian. Our general theory is also useful in studying the support recovery problem. To broaden the applicability, we further extend our main results to generalized linear models with convex loss functions. The effectiveness of our methods is demonstrated via simulation studies. Supplementary materials for this article are available online.

statistics & probability
A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models

Yang Ning,Han Liu

DOI: https://doi.org/10.48550/arXiv.1412.8765

2015-01-21

Abstract:We consider the problem of uncertainty assessment for low dimensional components in high dimensional models. Specifically, we propose a decorrelated score function to handle the impact of high dimensional nuisance parameters. We consider both hypothesis tests and confidence regions for generic penalized M-estimators. Unlike most existing inferential methods which are tailored for individual models, our approach provides a general framework for high dimensional inference and is applicable to a wide range of applications. From the testing perspective, we develop general theorems to characterize the limiting distributions of the decorrelated score test statistic under both null hypothesis and local alternatives. These results provide asymptotic guarantees on the type I errors and local powers of the proposed test. Furthermore, we show that the decorrelated score function can be used to construct point and confidence region estimators that are semiparametrically efficient. We also generalize this framework to broaden its applications. First, we extend it to handle high dimensional null hypothesis, where the number of parameters of interest can increase exponentially fast with the sample size. Second, we establish the theory for model misspecification. Third, we go beyond the likelihood framework, by introducing the generalized score test based on general loss functions. Thorough numerical studies are conducted to back up the developed theoretical results.

Machine Learning
Online Inference with Debiased Stochastic Gradient Descent

Ruijian Han,Lan Luo,Yuanyuan Lin,Jian Huang

DOI: https://doi.org/10.1093/biomet/asad046

IF: 3.0279

2023-07-27

Biometrika

Abstract:Summary We propose a debiased stochastic gradient descent algorithm for online statistical inference with high-dimensional data. Our approach combines the debiasing technique developed in high-dimensional statistics with the stochastic gradient descent algorithm. It can be used for efficiently constructing confidence intervals in an online fashion. Our proposed algorithm has several appealing aspects: first, as a one-pass algorithm, it reduces the time complexity; in addition, each update step requires only the current data together with the previous estimate, which reduces the space complexity. We establish the asymptotic normality of the proposed estimator under mild conditions on the sparsity level of the parameter and the data distribution. We conduct numerical experiments to demonstrate the proposed debiased stochastic gradient descent algorithm reaches nominal coverage probability. Furthermore, we illustrate our method with a high-dimensional text dataset.

statistics & probability,mathematical & computational biology,biology
A robust and efficient estimation and variable selection method for partially linear models with large-dimensional covariates

Hu Yang,Ning Li,Jing Yang

DOI: https://doi.org/10.1007/s00362-018-1013-1

2018-05-21

Statistical Papers

Abstract:In this paper, a new robust and efficient estimation approach based on local modal regression is proposed for partially linear models with large-dimensional covariates. We show that the resulting estimators for both parametric and nonparametric components are more efficient in the presence of outliers or heavy-tail error distribution, and as asymptotically efficient as the corresponding least squares estimators when there are no outliers and the error distribution is normal. We also establish the asymptotic properties of proposed estimators when the covariate dimension diverges at the rate of on\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$o\left( {\sqrt{n} } \right) \mathrm{{ }}$$\end{document}. To achieve sparsity and enhance interpretability, we develop a variable selection procedure based on SCAD penalty to select significant parametric covariates and show that the method enjoys the oracle property under mild regularity conditions. Moreover, we propose a practical modified MEM algorithm for the proposed procedures. Some Monte Carlo simulations and a real data are conducted to illustrate the finite sample performance of the proposed estimators. Finally, based on the idea of sure independence screening procedure proposed by Fan and Lv (J R Stat Soc 70:849–911, 2008), a robust two-step approach is introduced to deal with ultra-high dimensional data.

statistics & probability
Confidence Intervals for Low-Dimensional Parameters in High-Dimensional Linear Models

Cun-Hui Zhang,Stephanie S. Zhang

DOI: https://doi.org/10.48550/arXiv.1110.2563

2012-11-02

Abstract:The purpose of this paper is to propose methodologies for statistical inference of low-dimensional parameters with high-dimensional data. We focus on constructing confidence intervals for individual coefficients and linear combinations of several of them in a linear regression model, although our ideas are applicable in a much broad context. The theoretical results presented here provide sufficient conditions for the asymptotic normality of the proposed estimators along with a consistent estimator for their finite-dimensional covariance matrices. These sufficient conditions allow the number of variables to far exceed the sample size. The simulation results presented here demonstrate the accuracy of the coverage probability of the proposed confidence intervals, strongly supporting the theoretical results.

Methodology
Statistical inference for high dimensional regression via Constrained Lasso

Yun Yang

DOI: https://doi.org/10.48550/arXiv.1704.05098

2017-04-18

Abstract:In this paper, we propose a new method for estimation and constructing confidence intervals for low-dimensional components in a high-dimensional model. The proposed estimator, called Constrained Lasso (CLasso) estimator, is obtained by simultaneously solving two estimating equations---one imposing a zero-bias constraint for the low-dimensional parameter and the other forming an $\ell_1$-penalized procedure for the high-dimensional nuisance parameter. By carefully choosing the zero-bias constraint, the resulting estimator of the low dimensional parameter is shown to admit an asymptotically normal limit attaining the Cramér-Rao lower bound in a semiparametric sense. We propose a tuning-free iterative algorithm for implementing the CLasso. We show that when the algorithm is initialized at the Lasso estimator, the de-sparsified estimator proposed in van de Geer et al. [\emph{Ann. Statist.} {\bf 42} (2014) 1166--1202] is asymptotically equivalent to the first iterate of the algorithm. We analyse the asymptotic properties of the CLasso estimator and show the globally linear convergence of the algorithm. We also demonstrate encouraging empirical performance of the CLasso through numerical studies.

Methodology,Statistics Theory,Computation,Machine Learning
High-dimensional regression in practice: an empirical study of finite-sample prediction, variable selection and ranking

Fan Wang,Sach Mukherjee,Sylvia Richardson,Steven M. Hill

DOI: https://doi.org/10.1007/s11222-019-09914-9

IF: 2.3241

2019-12-19

Statistics and Computing

Abstract:Abstract Penalized likelihood approaches are widely used for high-dimensional regression. Although many methods have been proposed and the associated theory is now well developed, the relative efficacy of different approaches in finite-sample settings, as encountered in practice, remains incompletely understood. There is therefore a need for empirical investigations in this area that can offer practical insight and guidance to users. In this paper, we present a large-scale comparison of penalized regression methods. We distinguish between three related goals: prediction, variable selection and variable ranking. Our results span more than 2300 data-generating scenarios, including both synthetic and semisynthetic data (real covariates and simulated responses), allowing us to systematically consider the influence of various factors (sample size, dimensionality, sparsity, signal strength and multicollinearity). We consider several widely used approaches (Lasso, Adaptive Lasso, Elastic Net, Ridge Regression, SCAD, the Dantzig Selector and Stability Selection). We find considerable variation in performance between methods. Our results support a “no panacea” view, with no unambiguous winner across all scenarios or goals, even in this restricted setting where all data align well with the assumptions underlying the methods. The study allows us to make some recommendations as to which approaches may be most (or least) suitable given the goal and some data characteristics. Our empirical results complement existing theory and provide a resource to compare methods across a range of scenarios and metrics.

statistics & probability,computer science, theory & methods
A Constructive Approach to High-dimensional Regression

Jian Huang,Yuling Jiao,Yanyan Liu,Xiliang Lu

DOI: https://doi.org/10.48550/arxiv.1701.05128

2017-01-01

Abstract:We develop a constructive approach to estimating sparse, high-dimensional linear regression models. The approach is a computational algorithm motivated from the KKT conditions for the ℓ_0-penalized least squares solutions. It generates a sequence of solutions iteratively, based on support detection using primal and dual information and root finding. We refer to the algorithm as SDAR for brevity. Under a sparse Rieze condition on the design matrix and certain other conditions, we show that with high probability, the ℓ_2 estimation error of the solution sequence decays exponentially to the minimax error bound in O(√(J)log(R)) steps; and under a mutual coherence condition and certain other conditions, the ℓ_∞ estimation error decays to the optimal error bound in O(log(R)) steps, where J is the number of important predictors, R is the relative magnitude of the nonzero target coefficients. Computational complexity analysis shows that the cost of SDAR is O(np) per iteration. Moreover the oracle least squares estimator can be exactly recovered with high probability at the same cost if we know the sparsity level. We also consider an adaptive version of SDAR to make it more practical in applications. Numerical comparisons with Lasso, MCP and greedy methods demonstrate that SDAR is competitive with or outperforms them in accuracy and efficiency.
High-dimensional Model-assisted Inference for Local Average Treatment Effects with Instrumental Variables

Baoluo Sun,Zhiqiang Tan

DOI: https://doi.org/10.48550/arXiv.2009.09286

2020-09-20

Abstract:Consider the problem of estimating the local average treatment effect with an instrument variable, where the instrument unconfoundedness holds after adjusting for a set of measured covariates. Several unknown functions of the covariates need to be estimated through regression models, such as instrument propensity score and treatment and outcome regression models. We develop a computationally tractable method in high-dimensional settings where the numbers of regression terms are close to or larger than the sample size. Our method exploits regularized calibrated estimation, which involves Lasso penalties but carefully chosen loss functions for estimating coefficient vectors in these regression models, and then employs a doubly robust estimator for the treatment parameter through augmented inverse probability weighting. We provide rigorous theoretical analysis to show that the resulting Wald confidence intervals are valid for the treatment parameter under suitable sparsity conditions if the instrument propensity score model is correctly specified, but the treatment and outcome regression models may be misspecified. For existing high-dimensional methods, valid confidence intervals are obtained for the treatment parameter if all three models are correctly specified. We evaluate the proposed methods via extensive simulation studies and an empirical application to estimate the returns to education.

Methodology,Statistics Theory
Robust Inference for High-dimensional Linear Models with Heavy-tailed Errors via Partial Gini Covariance

Yilin Zhang,Songshan Yang,Yunan Wu,Lan Wang

2024-11-19

Abstract:This paper introduces the partial Gini covariance, a novel dependence measure that addresses the challenges of high-dimensional inference with heavy-tailed errors, often encountered in fields like finance, insurance, climate, and biology. Conventional high-dimensional regression inference methods suffer from inaccurate type I errors and reduced power in heavy-tailed contexts, limiting their effectiveness. Our proposed approach leverages the partial Gini covariance to construct a robust statistical inference framework that requires minimal tuning and does not impose restrictive moment conditions on error distributions. Unlike traditional methods, it circumvents the need for estimating the density of random errors and enhances the computational feasibility and robustness. Extensive simulations demonstrate the proposed method's superior power and robustness over standard high-dimensional inference approaches, such as those based on the debiased Lasso. The asymptotic relative efficiency analysis provides additional theoretical insight on the improved efficiency of the new approach in the heavy-tailed setting. Additionally, the partial Gini covariance extends to the multivariate setting, enabling chi-square testing for a group of coefficients. We illustrate the method's practical application with a real-world data example.

Methodology,Statistics Theory
The Impact of Serum Interleukin-17 on Chronic Hepatitis C and Its

SequelaeAhmed Hassan,Abeer Sharaf,Abd El-rehim,A. Ahmed,Nahla M Elsherbiny,N. A. El-Rehim,E. Hassan

DOI: https://doi.org/10.4172/2167-0889.1000163

2014-09-05

Abstract:Background: Recently, interleukin-17 (IL-17) cytokine family plays important roles in the host immunity in chronic inflammatory conditions, autoimmune and viral liver diseases. However, studies on the IL-17 role in the immunopathogenesis of hepatitis C virus (HCV) infection are limited. Objectives: to assess serum IL-17 levels in chronically HCV infected patients and their relations to the severity of liver disease. Patients and methods: The study included 200 chronically HCV infected patients; 100 chronic hepatitis C, 100 liver cirrhosis including 35 with hepatocellular carcinoma (HCC) and 30 controls. Serum IL-17 levels were quantified by ELISA. Results: Serum IL-17 levels were significantly higher in chronically HCV infected patients than controls and cirrhotics had the highest levels (P<0.001). These levels were positively related with inflammation grade and fibrosis stage. Serum IL-17 was significantly higher in HCC than controls. IL-17 was significantly correlated with prothrombin time, ALT, serum albumin, viral load, and alpha fetoprotein-L3. Conclusion: IL-17 levels were increased with increasing liver disease progression and chronicity. Thus, IL-17 may be an important biological marker for the immunopathogenesis of chronic hepatitis, liver fibrosis and HCC.
On Ridge Estimation in High-dimensional Rotationally Sparse Linear Regression

Libin Liang,Zhiqiang Tan

2024-05-02

Abstract:Recently, deep neural networks have been found to nearly interpolate training data but still generalize well in various applications. To help understand such a phenomenon, it has been of interest to analyze the ridge estimator and its interpolation limit in high-dimensional regression models. For this motivation, we study the ridge estimator in a rotationally sparse setting of high-dimensional linear regression, where the signal of a response is aligned with a small number, $d$, of covariates with large or spiked variances, compared with the remaining covariates with small or tail variances, \textit{after} an orthogonal transformation of the covariate vector. We establish high-probability upper and lower bounds on the out-sample and in-sample prediction errors in two distinct regimes depending on the ratio of the effective rank of tail variances over the sample size $n$. The separation of the two regimes enables us to exploit relevant concentration inequalities and derive concrete error bounds without making any oracle assumption or independent components assumption on covariate vectors. Moreover, we derive sufficient and necessary conditions which indicate that the prediction errors of ridge estimation can be of the order $O(\frac{d}{n})$ if and only if the gap between the spiked and tail variances are sufficiently large. We also compare the orders of optimal out-sample and in-sample prediction errors and find that, remarkably, the optimal out-sample prediction error may be significantly smaller than the optimal in-sample one. Finally, we present numerical experiments which empirically confirm our theoretical findings.

Statistics Theory
Regret Minimization and Statistical Inference in Online Decision Making with High-dimensional Covariates

Congyuan Duan,Wanteng Ma,Jiashuo Jiang,Dong Xia

2024-11-10

Abstract:This paper investigates regret minimization, statistical inference, and their interplay in high-dimensional online decision-making based on the sparse linear context bandit model. We integrate the $\varepsilon$-greedy bandit algorithm for decision-making with a hard thresholding algorithm for estimating sparse bandit parameters and introduce an inference framework based on a debiasing method using inverse propensity weighting. Under a margin condition, our method achieves either $O(T^{1/2})$ regret or classical $O(T^{1/2})$-consistent inference, indicating an unavoidable trade-off between exploration and exploitation. If a diverse covariate condition holds, we demonstrate that a pure-greedy bandit algorithm, i.e., exploration-free, combined with a debiased estimator based on average weighting can simultaneously achieve optimal $O(\log T)$ regret and $O(T^{1/2})$-consistent inference. We also show that a simple sample mean estimator can provide valid inference for the optimal policy's value. Numerical simulations and experiments on Warfarin dosing data validate the effectiveness of our methods.

Machine Learning

JAAA from a geographic perspective.

Sparsified simultaneous confidence intervals for high-dimensional linear models

Efficient Inference on High-Dimensional Linear Models with Missing Outcomes

Inference for High-Dimensional Linear Expectile Regression with De-Biasing Method

Statistical Inference and Large-scale Multiple Testing for High-dimensional Regression Models

Model-Free Statistical Inference on High-Dimensional Data

Inference in High-Dimensional Linear Measurement Error Models

Doubly Robust Semiparametric Inference Using Regularized Calibrated Estimation with High-dimensional Data

Simultaneous Inference for High-Dimensional Linear Models

A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models

Online Inference with Debiased Stochastic Gradient Descent

A robust and efficient estimation and variable selection method for partially linear models with large-dimensional covariates

Confidence Intervals for Low-Dimensional Parameters in High-Dimensional Linear Models

Statistical inference for high dimensional regression via Constrained Lasso

High-dimensional regression in practice: an empirical study of finite-sample prediction, variable selection and ranking

A Constructive Approach to High-dimensional Regression

High-dimensional Model-assisted Inference for Local Average Treatment Effects with Instrumental Variables

Robust Inference for High-dimensional Linear Models with Heavy-tailed Errors via Partial Gini Covariance

The Impact of Serum Interleukin-17 on Chronic Hepatitis C and Its

On Ridge Estimation in High-dimensional Rotationally Sparse Linear Regression

Regret Minimization and Statistical Inference in Online Decision Making with High-dimensional Covariates