Abstract:Many data sets exhibit a natural group structure due to contextual similarities or high correlations of variables, such as lipid markers that are interrelated based on biochemical principles. Knowledge of such groupings can be used through bi‐level selection methods to identify relevant feature groups and highlight their predictive members. One of the best known approaches of this kind combines the classical Least Absolute Shrinkage and Selection Operator (LASSO) with the Group LASSO, resulting in the Sparse Group LASSO. We propose the Sparse Group Penalty (SGP) framework, which allows for a flexible combination of different SGL‐style shrinkage conditions. Analogous to SGL, we investigated the combination of the Smoothly Clipped Absolute Deviation (SCAD), the Minimax Concave Penalty (MCP) and the Exponential Penalty (EP) with their group versions, resulting in the Sparse Group SCAD, the Sparse Group MCP, and the novel Sparse Group EP (SGE). Those shrinkage operators provide refined control of the effect of group formation on the selection process through a tuning parameter. In simulation studies, SGPs were compared with other bi‐level selection methods (Group Bridge, composite MCP, and Group Exponential LASSO) for variable and group selection evaluated with the Matthews correlation coefficient. We demonstrated the advantages of the new SGE in identifying parsimonious models, but also identified scenarios that highlight the limitations of the approach. The performance of the techniques was further investigated in a real‐world use case for the selection of regulated lipids in a randomized clinical trial.

Versatile Descent Algorithms for Group Regularization and Variable Selection in Generalized Linear Models

A Group Adaptive Elastic-Net Approach for Variable Selection in High-Dimensional Linear Regression

Grouped Variable Selection with Discrete Optimization: Computational and Statistical Perspectives

Bayesian Model Selection with Latent Group-Based Effects and Variances with the R Package slgf

A Fast and Scalable Pathwise-Solver for Group Lasso and Elastic Net Penalized Regression via Block-Coordinate Descent

Regularization paths for generalized linear models via coordinate descent

Better prediction by use of co-data: Adaptive group-regularized ridge regression

Generalized Exponentiated Gradient Algorithms and Their Application to On-Line Portfolio Selection

Automatic Variable Selection For Longitudinal Generalized Linear Models

Generalized Newton–Raphson Algorithm for High Dimensional LASSO Regression

Consistent Tuning Parameter Selection in High-Dimensional Group-Penalized Regression

Robust Grouped Variable Selection Using Distributionally Robust Optimization

Sparse Group Penalties for bi‐level variable selection

Adaptive Noisy Data Augmentation for Regularized Estimation and Inference in Generalized Linear Models

Regularization and variable selection for data with interdependent structures

Bi-level variable selection via adaptive sparse group Lasso

Robust Variable Selection for Generalized Linear Models with A Diverging Number of Parameters

Exploiting regularity in sparse Generalized Linear Models

Consistent group selection in high-dimensional linear regression

Adaptive debiased SGD in high-dimensional GLMs with streaming data