Abstract:The problems of Lasso regression and optimal design of experiments share a critical property: their optimal solutions are typically \emph{sparse}, i.e., only a small fraction of the optimal variables are non-zero. Therefore, the identification of the support of an optimal solution reduces the dimensionality of the problem and can yield a substantial simplification of the calculations. It has recently been shown that linear regression with a \emph{squared} $\ell_1$-norm sparsity-inducing penalty is equivalent to an optimal experimental design problem. In this work, we use this equivalence to derive safe screening rules that can be used to discard inessential samples. Compared to previously existing rules, the new tests are much faster to compute, especially for problems involving a parameter space of high dimension, and can be used dynamically within any iterative solver, with negligible computational overhead. Moreover, we show how an existing homotopy algorithm to compute the regularization path of the lasso method can be reparametrized with respect to the squared $\ell_1$-penalty. This allows the computation of a Bayes $c$-optimal design in a finite number of steps and can be several orders of magnitude faster than standard first-order algorithms. The efficiency of the new screening rules and of the homotopy algorithm are demonstrated on different examples based on real data.

A Safe Feature Screening Rule for Rank Lasso

A Safe Reinforced Feature Screening Strategy for Lasso Based on Feasible Solutions

Safe Feature Identification Rule for Fused Lasso by An Extra Dual Variable

Safe Feature Screening Rules for the Regularized Huber Regression

Look-Ahead Screening Rules for the Lasso

A Safe Screening Rule for Sparse Logistic Regression

A Novel Self-Weighted Lasso and Its Safe Screening Rule

Gap safe screening rules for sparse-group lasso

Screening Rules for Lasso with Non-Convex Sparse Regularizers

Mind the duality gap: safer rules for the Lasso

Gap safe screening rules for sparsity enforcing penalties

L1-Norm Quantile Regression Screening Rule Via the Dual Circumscribed Sphere

Feature Screening Strategy for Non-Convex Sparse Logistic Regression with Log Sum Penalty

A Safe Feature Elimination Rule for L1-Regularized Logistic Regression.

Gap safe screening rules for sparse multi-task and multi-class models

Safe Feature Elimination for the LASSO and Sparse Supervised Learning Problems

Fast Screening Rules for Optimal Design via Quadratic Lasso Reformulation

Safe Screening for Logistic Regression with $\ell_0$-$\ell_2$ Regularization

Strong rules for discarding predictors in lasso-type problems

Fast OSCAR and OWL Regression via Safe Screening Rules