Abstract:Sparse optimization problems are ubiquitous in many fields such as statistics, signal/image processing and machine learning. This has led to the birth of many iterative algorithms to solve them. A powerful strategy to boost the performance of these algorithms is known as safe screening: it allows the early identification of zero coordinates in the solution, which can then be eliminated to reduce the problem's size and accelerate convergence. In this work, we extend the existing Gap Safe screening framework by relaxing the global strong-concavity assumption on the dual cost function. Instead, we exploit local regularity properties, that is, strong concavity on well-chosen subsets of the domain. The non-negativity constraint is also integrated to the existing framework. Besides making safe screening possible to a broader class of functions that includes beta-divergences (e.g., the Kullback-Leibler divergence), the proposed approach also improves upon the existing Gap Safe screening rules on previously applicable cases (e.g., logistic regression). The proposed general framework is exemplified by some notable particular cases: logistic function, beta = 1.5 and Kullback-Leibler divergences. Finally, we showcase the effectiveness of the proposed screening rules with different solvers (coordinate descent, multiplicative-update and proximal gradient algorithms) and different data sets (binary classification, hyperspectral and count data).

Gap safe screening rules for sparsity enforcing penalties

Gap safe screening rules for sparse multi-task and multi-class models

Gap safe screening rules for sparse-group lasso

Gap Safe screening rules for faster complex-valued multi-task group Lasso

Look-Ahead Screening Rules for the Lasso

Mind the duality gap: safer rules for the lasso

Expanding boundaries of Gap Safe screening

A Safe Screening Rule for Sparse Logistic Regression

Screening Rules for Lasso with Non-Convex Sparse Regularizers

Feature Screening Strategy for Non-Convex Sparse Logistic Regression with Log Sum Penalty

From safe screening rules to working sets for faster lasso-type solvers

Safe Screening for Logistic Regression with $\ell_0$-$\ell_2$ Regularization

Faithful Variable Screening for High-Dimensional Convex Regression

A Safe Feature Screening Rule for Rank Lasso

Variable Screening for Sparse Online Regression.

Fast Screening Rules for Optimal Design via Quadratic Lasso Reformulation

Exploiting regularity in sparse Generalized Linear Models

Strong screening rules for group-based SLOPE models

Screening for Sparse Online Learning

Strong rules for discarding predictors in lasso-type problems