Abstract:This note extends the results of classical parametric statistics like Fisher and Wilks theorem to modern setups with a high or infinite parameter dimension, limited sample size, and possible model misspecification. We consider a special class of stochastically linear smooth (SLS) models satisfying three major conditions: the stochastic component of the log-likelihood is linear in the model parameter and the expected log-likelihood is a smooth and concave function. For the penalized maximum likelihood estimators (pMLE), we establish three types of results: (1) concentration in a small vicinity of the ``truth''; (2) Fisher and Wilks expansions; (3) risk bounds. In all results, the remainder is given explicitly and can be evaluated in terms of the effective sample size and effective parameter dimension which allows us to identify the so-called \emph{critical parameter dimension}. The results are also dimension and coordinate-free. The obtained finite sample expansions are of special interest because they can be used not only for obtaining the risk bounds but also for inference, studying the asymptotic distribution, analysis of resampling procedures, etc. The main tool for all these expansions is the so-called ``basic lemma'' about linearly perturbed optimization. Despite their generality, all the presented bounds are nearly sharp and the classical asymptotic results can be obtained as simple corollaries. Our results indicate that the use of advanced fourth-order expansions allows to relax the critical dimension condition $ \mathbb{p}^{3} \ll n $ from Spokoiny (2023a) to $ \mathbb{p}^{3/2} \ll n $. Examples for classical models like logistic regression, log-density and precision matrix estimation illustrate the applicability of general results.

Provable More Data Hurt in High Dimensional Least Squares Estimator

Robust Estimation and Shrinkage in Ultrahigh Dimensional Expectile Regression with Heavy Tails and Variance Heterogeneity

Expectile regression for analyzing heteroscedasticity in high dimension

Asymptotic Normality and Confidence Intervals for Prediction Risk of the Min-Norm Least Squares Estimator.

Inference for High-Dimensional Linear Expectile Regression with De-Biasing Method

Penalized Independence Rule for Testing High-Dimensional Hypotheses

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

Effects of data dimension on empirical likelihood

Model-Free Statistical Inference on High-Dimensional Data

Rank-Constrained Least-Squares: Prediction and Inference

Finite sample expansions and risk bounds in high-dimensional SLS models

High-dimensional regression in practice: an empirical study of finite-sample prediction, variable selection and ranking

Non-separable Models with High-dimensional Data

Prediction Risk and Estimation Risk of the Ridgeless Least Squares Estimator under General Assumptions on Regression Errors

Empirical Likelihood for High Dimensional Data

High-probability minimax lower bounds

L G ] 2 5 O ct 2 01 8 l 1-regression with Heavy-tailed Distributions

Estimation and Inference in Ultrahigh Dimensional Partially Linear Single-Index Models

CONFIDENCE INTERVALS FOR HIGH-DIMENSIONAL LINEAR REGRESSION: MINIMAX RATES AND ADAPTIVITY

High-dimensional Measurement Error Models for Lipschitz Loss

Statistical Inference from High Dimensional Data