Abstract:Nonlinear regression problem is one of the most popular and important statistical tasks. The first methods like least squares estimation go back to Gauss and Legendre. Recent models and developments in statistics and machine learning like Deep Neuronal Networks or Bayesian methods for nonlinear PDE stimulate new research in this direction which has to address the important issues and challenges of modern statistical inference such as huge complexity and parameter dimension of the model, limited samples size, lack of convexity and identifiability among many others. Classical results of nonparametric statistics in terms of rate of convergence fail to explain the mentioned issues because of the curse of dimensionality problem. This note offers a general approach to studying a nonlinear regression problem which enables one to derive finite sample expansions for the loss of the penalized maximum likelihood estimation (pMLE) with explicit error guarantees and obtain sharp loss and risk bounds. An important step of the study called calming allows to make the objective function stochastically linear by extending the parameter space and to reduce the original problem to semiparametric estimation with a special stochastically linear structure. Such models are studied in this paper in the full generality, the results provide finite sample expansions and risk bounds for the full and target parameters. In all results, the remainder is given explicitly and can be evaluated in terms of the effective sample size and effective parameter dimension which allows us to identify the so-called critical parameter dimension. The results are also dimension and coordinate-free. Despite generality, all the presented bounds are nearly sharp and the classical asymptotic results can be obtained as simple corollaries. The obtained general results are specified to nonlinear smooth regression.

Statistical inference on nonparametric regression model with approximation of Fourier series function: Estimation and hypothesis testing

A method for parameter hypothesis testing in nonparametric regression with Fourier series approach

Testing Parallelism of Nonparametric Regression Curves

Consistent Specification Testing Via Nonparametric Series Regression

Statistical Inference of the Class of Nonparametric Tests for the Panel Count and Current Status Data from the Perspective of the Saddlepoint Approximation

Empirical-Likelihood-Based Inference for Partially Linear Models

Semiparametric Regression Based on Quadratic Inference Function for Multivariate Failure Time Data with Auxiliary Information

Statistical inference in functional semiparametric spatial autoregressive model

Nonparametric Trend Estimation in Functional Time Series with Application to Annual Mortality Rates

Adaptive Optimal Nonparametric Regression and Density Estimation Based on Fourier-Legendre Expansion

Inference in the nonparametric stochastic frontier model

Statistical inference for semiparametric varying-coefficient partially linear models with error-prone linear covariates

Non-linear regression models for Approximate Bayesian Computation

Modeling Various Survival Distributions using a Nonparametric Hypothesis Testing Based on Laplace Transform Approach with Some Real Applications

On a new class of tests for the Pareto distribution using Fourier methods

Nonparametric Bayesian inference for mean residual life functions in survival analysis

Finite sample inference in nonlinear regression estimation

Efficient Estimation of Nonlinear Finite Population Parameters Using Nonparametrics

Statistical inference for partially linear stochastic models with heteroscedastic errors.

An interval estimator of error variance in nonparametric regression model

Survey of non-parametric estimation methods