Abstract:In this paper, we study Lipschitz continuity of the solution mappings of regularized least-squares problems for which the convex regularizers have (Fenchel) conjugates that are $\mathcal{C}^2$-cone reducible. Our approach, by using Robinson's strong regularity on the dual problem, allows us to obtain new characterizations of Lipschitz stability that rely solely on first-order information, thus bypassing the need to explore second-order information (curvature) of the regularizer. We show that these solution mappings are automatically Lipschitz continuous around the points in question whenever they are locally single-valued. We leverage our findings to obtain new characterizations of full stability and tilt stability for a broader class of convex additive-composite problems.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the Lipschitz stability of the regularized least - squares problem, especially when the Fenchel conjugate function of the regularization term is C2 - cone - reducible. Specifically, the author studies whether the solution mapping of these problems is single - valued and Lipschitz continuous under small perturbations. ### Research Background and Motivation Stability (or sensitivity) analysis in optimization problems studies how the optimal solution (or optimal value) changes when the data (parameters) defining the problem change. This field has a long history, mainly laid by the pioneering work of Kojima [17, 18] and Robinson [28, 29, 30], etc. In recent years, with the wide application of non - smooth convex optimization problems in machine learning and statistical estimation, the stability analysis of such problems has received attention again. ### Core Problems of the Paper This paper pays special attention to the regularized least - squares problem in the following form: \[ \min_{x\in X}\frac{1}{2\mu}\|Ax - b\|^{2}+g(x), \] where $A:X\rightarrow Y$ is a linear operator between two Euclidean spaces, $b\in Y$, $\mu>0$ is a tuning parameter, and $g:X\rightarrow\mathbb{R}\cup\{+\infty\}$ is a closed, proper, convex function (usually called a regularization term in machine learning). When regarding the triple $(A, b, \mu)$ as parameters, it is important to study the stability of the solution mapping $S(A, b, \mu)$ of problem (1) when the initial data $(A, b, \mu)$ has small perturbations. ### Main Contributions 1. **Lipschitz Stability Conditions**: By using Robinson's strong regularity theory, the author gives new characterizations of Lipschitz stability, which only depend on first - order information, thus avoiding the exploration of the curvature (second - order information) of the regularization term. 2. **Single - Valuedness and Lipschitz Continuity**: The author proves that the solution mapping $S(A, b, \mu)$ is single - valued and Lipschitz continuous under certain conditions. 3. **Extension to Broader Problems**: The author uses these results to obtain new characterizations of complete stability and tilt stability for broader convex additive composite problems. ### Methodology The author adopts the strong regularity theory and achieves this goal by studying the dual problem. Specifically, they consider the dual problem in the following form: \[ D(A, b, \mu, v):\min_{y\in Y}\frac{\mu}{2}\|y\|^{2}-\langle b, y\rangle+g^{*}(A^{*}y + v). \] By studying the strong regularity of the dual problem, the author can derive the Lipschitz stability conditions of the solution mapping of the original problem. The advantage of this method is that it can simplify the second - order calculation requirements for the regularization term, making the conditions easier to verify. ### Conclusion The main contribution of this paper is to provide a new method to study the Lipschitz stability of the regularized least - squares problem, especially when the Fenchel conjugate function of the regularization term is C2 - cone - reducible. By introducing the strong regularity theory, the author not only simplifies the expression of the conditions but also expands the application range of these results, covering broader convex optimization problems.

Lipschitz stability of least-squares problems regularized by functions with $\mathcal{C}^2$-cone reducible conjugates

Decomposition Method for Lipschitz Stability of General LASSO-type Problems

STABLE LAGRANGE DUALITIES FOR ROBUST CONICAL PROGRAMMING

Exact Lipschitz Regularization of Convex Optimization Problems

Stable and Total Fenchel Duality for Convex Optimization Problems in Locally Convex Spaces

Stability Criteria and Calculus Rules via Conic Contingent Coderivatives in Banach Spaces

Lipschitz continuity of solution multifunctions of extended $\ell_1$ regularization problems

Convergence Criterion and Convergence Ball of the King-Werner Method under the Radius Lipschitz Condition

On Convergence Lemma and Convergence Stability for Piecewise Analytic Functions

On Euler-Dierkes-Huisken variational problem

Relative Regularity Conditions and Linear Regularity Properties for Split Feasibility Problems in Normed Linear Spaces

Lipschitz regularity for manifold-constrained ROF elliptic systems

Lipschitz regularity for solutions of a general class of elliptic equations

Geometric characterizations of Lipschitz stability for convex optimization problems

Optimality conditions and Lipschitz stability for non-smooth semilinear elliptic optimal control problems with sparse controls

An Analysis of Tikhonov Regularization for Nonlinear Ill-Posed Problems under a General Smoothness Assumption

Characterizing Parametric and Convergence Stability in Nonconvex and Nonsmooth Optimizations: A Geometric Approach

Lipschitz regularity of a weakly coupled vectorial almost-minimizers for the $p$-Laplacian

Lipschitz regularity for orthotropic functionals with nonstandard growth conditions

Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert Spaces

On Linear Convergence in Smooth Convex-Concave Bilinearly-Coupled Saddle-Point Optimization: Lower Bounds and Optimal Algorithms