Abstract:We propose a novel high-dimensional linear regression estimator: the Discrete Dantzig Selector, which minimizes the number of nonzero regression coefficients subject to a budget on the maximal absolute correlation between the features and residuals. Motivated by the significant advances in integer optimization over the past 10-15 years, we present a Mixed Integer Linear Optimization (MILO) approach to obtain certifiably optimal global solutions to this nonconvex optimization problem. The current state of algorithmics in integer optimization makes our proposal substantially more computationally attractive than the least squares subset selection framework based on integer quadratic optimization, recently proposed in [8] and the continuous nonconvex quadratic optimization framework of [33]. We propose new discrete first-order methods, which when paired with state-of-the-art MILO solvers, lead to good solutions for the Discrete Dantzig Selector problem for a given computational budget. We illustrate that our integrated approach provides globally optimal solutions in significantly shorter computation times, when compared to off-the-shelf MILO solvers. We demonstrate both theoretically and empirically that in a wide range of regimes the statistical properties of the Discrete Dantzig Selector are superior to those of popular $\ell_{1}$-based approaches. We illustrate that our approach can handle problem instances with p = 10,000 features with certifiable optimality making it a highly scalable combinatorial variable selection approach in sparse linear modeling.

High-dimensional stochastic optimization with the generalized Dantzig estimator

Generalization of l1 constraints for high dimensional regression problems

Two step estimations via the Dantzig selector for models of stochastic processes with high-dimensional parameters

The constrained Dantzig selector with enhanced consistency

Generalized Dantzig Selector: Application to the k-support norm

The Discrete Dantzig Selector: Estimating Sparse Linear Models via Mixed Integer Linear Optimization

Simultaneous analysis of Lasso and Dantzig selector

Parallelism, uniqueness, and large-sample asymptotics for the Dantzig selector

Inference for High-Dimensional Linear Expectile Regression with De-Biasing Method

Variable Selection and Minimax Prediction in High-dimensional Functional Linear Model

The Dantzig selector for a linear model of diffusion processes

L2 Boosting on generalized Hoeffding decomposition for dependent variables. Application to Sensitivity Analysis

High-dimensional variable selection with heterogeneous signals: A precise asymptotic perspective

Computing Estimators of Dantzig Selector type via Column and Constraint Generation

Transductive versions of the LASSO and the Dantzig Selector

Linear and Conic Programming Estimators in High-Dimensional Errors-in-variables Models

Dual Lasso Selector

Confidence regions for high-dimensional generalized linear models under sparsity

Covariate Selection in High-Dimensional Generalized Linear Models With Measurement Error

Inference for High-Dimensional Sparse Econometric Models

On the Computational Complexity of Private High-dimensional Model Selection