Abstract:A popular approach for estimating an unknown signal from noisy, linear measurements is via solving a so called \emph{regularized M-estimator}, which minimizes a weighted combination of a convex loss function and of a convex (typically, non-smooth) regularizer. We accurately predict the squared error performance of such estimators in the high-dimensional proportional regime. The random measurement matrix is assumed to have entries iid Gaussian, only minimal and rather mild regularity conditions are imposed on the loss function, the regularizer, and on the noise and signal distributions. We show that the error converges in probability to a nontrivial limit that is given as the solution to a minimax convex-concave optimization problem on four scalar optimization variables. We identify a new summary parameter, termed the Expected Moreau envelope to play a central role in the error characterization. The \emph{precise} nature of the results permits an accurate performance comparison between different instances of regularized M-estimators and allows to optimally tune the involved parameters (e.g. regularizer parameter, number of measurements). The key ingredient of our proof is the \emph{Convex Gaussian Min-max Theorem} (CGMT) which is a tight and strengthened version of a classical Gaussian comparison inequality that was proved by Gordon in 1988.

Parameter Estimation on Random Regular Matrix in Regression Model Error Compensation Semi Parametric Method

Efficient Estimation for Error Component Seemingly Unrelated Nonparametric Regression Models

Estimation of Varying-Coefficient Partially Linear Model with Measurement Error in the Parametric Part

Parameter Estimability Analysis for Process System with Finite Measurement Information

Low-rank matrix estimation via nonconvex spectral regularized methods in errors-in-variables matrix regression

Algorithmic Regularization in Model-free Overparametrized Asymmetric Matrix Factorization

Stochastic Restricted Estimation in Partially Linear Measurement Error Models

Low-rank matrix estimation in multi-response regression with measurement errors: Statistical and computational guarantees.

Nonparametric Regression Estimation with General Parametric Error Covariance: a More Efficient Two-Step Estimator

The Numerical Simulation of Improving Parameter Estimation by Instrumental Variable Method

A Perturbation Method for Inference on Regularized Regression Estimates

Parametric modal regression with error in covariates

Precise Error Analysis of Regularized M-estimators in High-dimensions

Parameter Estimation for Multivariate Nonlinear Stochastic Differential Equation Models: A Comparison Study

Regularized Nonlinear Regression with Dependent Errors and its Application to a Biomechanical Model

A New Two-Parameter Estimator in Linear Regression

Restricted Estimation in Partially Linear Errors-In-Variables Models

On a Biased Prediction Based on Optimal Mean Square Error Criterion

A Parametric Bootstrap Approach for a One-Way Error Component Regression Model with Measurement Errors

The Bayes Estimation of Parameters in Semi-Parametric Regression Model

Estimated Theory of Semiparametric Regression Model With Heteroscedasticity