Abstract:Major progress has been made in the previous decade to characterize the asymptotic behavior of regularized M-estimators in high-dimensional regression problems in the proportional asymptotic regime where the sample size $n$ and the number of features $p$ are increasing simultaneously such that $n/p\to \delta \in(0,\infty)$, using powerful tools such as Approximate Message Passing or the Convex Gaussian Min-Max Theorem (CGMT). The asymptotic error and behavior of the regularized M-estimator is then typically described by a system of nonlinear equations with a few scalar unknowns, and the solution to this system precisely characterizes the asymptotic error. Application of the CGMT and related machinery requires the existence and uniqueness of a solution to this low-dimensional system of equations or to a related scalar convex minimization problem. This paper resolves the question of existence and uniqueness of solution to this low-dimensional system for the case of linear models with independent additive noise, when both the data-fitting loss function and regularizer are separable and convex. Such existence result was previously known under strong convexity or for specific estimators such as the Lasso. The main idea behind this existence result is inspired by an argument developed by Montanari et al. [2023] and Celentano et al. [2023] in different contexts: By constructing an ad-hoc convex minimization problem in an infinite dimensional Hilbert space, the existence of the Lagrange multiplier for this optimization problem makes it possible to construct explicit solutions to the low-dimensional system of interest. The conditions under which we derive this existence result exactly correspond to the side of the phase transition where perfect recovery $\hat{x} = x_0$ fails, so that these conditions are optimal.

Sign consistent estimation in a sparse Poisson model

Sparse inference in Poisson Log-Normal model by approximating the L0-norm

The Sparse Poisson Means Model

On Sparse Estimation for Semiparametric Linear Transformation Models

Sparse Estimation Via Lower-Order Penalty Optimization Methods in High-Dimensional Linear Regression.

Sparse Poisson Regression with Penalized Weighted Score Function

Sparse Estimation Via ℓ_q Optimization Method in High-Dimensional Linear Regression

Penalized robust estimators in logistic regression with applications to sparse models

Poisson intensity parameter estimation for stationary Gibbs point processes of finite interaction range

On the Conditions of Sparse Parameter Estimation via Log-Sum Penalty Regularization.

Posterior contraction rates in a sparse non-linear mixed-effects model

Statistical inference in sparse high-dimensional additive models

Estimation and confidence sets for sparse normal mixtures

Sparse permutation invariant covariance estimation

On High dimensional Poisson models with measurement error: hypothesis testing for nonlinear nonconvex optimization

Sparse Estimation from Sign Measurements with General Sensing Matrix Perturbation

Sparse estimation for generalized exponential marked Hawkes process

Provably Efficient Posterior Sampling for Sparse Linear Regression via Measure Decomposition

Importance sampling-based gradient method for dimension reduction in Poisson log-normal model

A Constrained L1 Minimization Approach to Sparse Precision Matrix Estimation

Existence of solutions to the nonlinear equations characterizing the precise error of M-estimators