Abstract:Accompanied with the increasing popularity of linear regression approaches, most of the existing minimization problems are related with several convex measurements, e.g., $\ell _{1}$ / $\ell _{2}$ / $\ell _{2,1}$ -norm of a vector and $L_{1}$ / $L_{2,1}$ /Frobenius/nuclear norm of a matrix, where the regularized function and the loss function are usually studied for two objective terms case by case, respectively. To address this issue, this work combines these linear regression problems into a unified expression framework by employing an adaptive and flexible function, in which we need to choose different variable elements and adjust an inner parameter, properly. Besides this, they are equipped with some corresponding relationships and their interesting properties. Intuitively speaking, the proposed framework can generalize several traditional linear regression formulations and even more complex ones into an extended representation. For further optimizations, an iteratively re-weighted penalty solution (IRwPS) is devised without any inner loops, making the iteration programming easy to perform. Meanwhile, the theoretical results are provided for guaranteeing that the mathematical convergence analysis is solid and meaningful. Finally, by performing real-world applications in supervised, unsupervised, and semi-supervised tasks, numerical experiments are conducted to validate the theoretical properties and the superiority over some of the state-of-the-art.

From Linear to Linearizable Optimization: A Novel Framework with Applications to Stationary and Non-stationary DR-submodular Optimization

A Fast Algorithm For Maximizing A Non-Monotone Dr-Submodular Integer Lattice Function

Unified Projection-Free Algorithms for Adversarial DR-Submodular Optimization

Bridging the Gap Between General and Down-Closed Convex Sets in Submodular Maximization

A Unified Algorithmic Framework for Distributed Composite Optimization.

Fast Semidifferential-based Submodular Function Optimization

A Unified Framework for Analyzing Meta-algorithms in Online Convex Optimization

Quadratic Decomposable Submodular Function Minimization: Theory and Practice (Computation and Analysis of PageRank over Hypergraphs)

Constrained Submodular Maximization via New Bounds for DR-Submodular Functions

Linear Submodular Maximization with Bandit Feedback

Online non-monotone diminishing return submodular maximization in the bandit setting

Linearization Algorithms for Fully Composite Optimization

Online DR-Submodular Maximization: Minimizing Regret and Constraint Violation

Maximizing Monotone DR-submodular Continuous Functions by Derivative-free Optimization

On Unconstrained Quasi-Submodular Function Optimization

Discrete nonlinear functions: formulations and applications in retail revenue management

A Smooth Primal-Dual Optimization Framework for Nonsmooth Composite Convex Minimization

Nonlinear Distributionally Robust Optimization

Non-Smooth, Hölder-Smooth, and Robust Submodular Maximization

Incorporating Linear Regression Problems into an Adaptive Framework with Feasible Optimizations.

Linear convergence of a dual optimization formulation for distributed optimization on directed graphs with unreliable communications