Abstract:Accompanied with the increasing popularity of linear regression approaches, most of the existing minimization problems are related with several convex measurements, e.g., $\ell _{1}$ / $\ell _{2}$ / $\ell _{2,1}$ -norm of a vector and $L_{1}$ / $L_{2,1}$ /Frobenius/nuclear norm of a matrix, where the regularized function and the loss function are usually studied for two objective terms case by case, respectively. To address this issue, this work combines these linear regression problems into a unified expression framework by employing an adaptive and flexible function, in which we need to choose different variable elements and adjust an inner parameter, properly. Besides this, they are equipped with some corresponding relationships and their interesting properties. Intuitively speaking, the proposed framework can generalize several traditional linear regression formulations and even more complex ones into an extended representation. For further optimizations, an iteratively re-weighted penalty solution (IRwPS) is devised without any inner loops, making the iteration programming easy to perform. Meanwhile, the theoretical results are provided for guaranteeing that the mathematical convergence analysis is solid and meaningful. Finally, by performing real-world applications in supervised, unsupervised, and semi-supervised tasks, numerical experiments are conducted to validate the theoretical properties and the superiority over some of the state-of-the-art.

Algorithm for Optimum Linear Regression

An efficient optimization approach for best subset selection in linear regression, with application to model selection and fitting in autoregressive time-series

Subset selection for multiple linear regression via optimization

Variable Selection with Stepwise and Best Subset Approaches.

Optimal Feature Selection for Sparse Linear Discriminant Analysis and Its Applications in Gene Expression Data

Residuals and Regression Diagnostics: Focusing on Logistic Regression

A Variance Minimization Criterion to Feature Selection Using Laplacian Regularization

G-Optimal Feature Selection with Laplacian regularization

Comprehensive Stepwise Selection for Logistic Regression

Optimal Variable Step-Size Lms Model and Algorithm with Independence Assumption

Optimal designs for comparing several regression curves

Active Learning Query Strategies for Linear Regression Based on Efficient Global Optimization

A mathematical programming approach for integrated multiple linear regression subset selection and validation

Incorporating Linear Regression Problems into an Adaptive Framework with Feasible Optimizations.

Best Subset Solution Path for Linear Dimension Reduction Models using Continuous Optimization

Optimal Subsampling for Large Sample Logistic Regression

COMBSS: best subset selection via continuous optimization

Optimal rate of the regularized regression learning algorithm

Globally Optimal Linear Model Fitting with Unit-Norm Constraint

CVXSADes: a stochastic algorithm for constructing optimal exact regression designs with single or multiple objectives

Optimal Designs in Sparse Linear Models