Abstract:Nonlinear regression problem is one of the most popular and important statistical tasks. The first methods like least squares estimation go back to Gauss and Legendre. Recent models and developments in statistics and machine learning like Deep Neuronal Networks or Bayesian methods for nonlinear PDE stimulate new research in this direction which has to address the important issues and challenges of modern statistical inference such as huge complexity and parameter dimension of the model, limited samples size, lack of convexity and identifiability among many others. Classical results of nonparametric statistics in terms of rate of convergence fail to explain the mentioned issues because of the curse of dimensionality problem. This note offers a general approach to studying a nonlinear regression problem which enables one to derive finite sample expansions for the loss of the penalized maximum likelihood estimation (pMLE) with explicit error guarantees and obtain sharp loss and risk bounds. An important step of the study called calming allows to make the objective function stochastically linear by extending the parameter space and to reduce the original problem to semiparametric estimation with a special stochastically linear structure. Such models are studied in this paper in the full generality, the results provide finite sample expansions and risk bounds for the full and target parameters. In all results, the remainder is given explicitly and can be evaluated in terms of the effective sample size and effective parameter dimension which allows us to identify the so-called critical parameter dimension. The results are also dimension and coordinate-free. Despite generality, all the presented bounds are nearly sharp and the classical asymptotic results can be obtained as simple corollaries. The obtained general results are specified to nonlinear smooth regression.

Exact Nonparametric Inference for Detection of Nonlinear Determinism

The Quadratic Inference Functions in Measurement Error Model for Longitudinal Data

Nonparametric Rank Test for Nonlinearity Detection

Real-time Detector for Time-variant Oscillation with Modified Intrinsic Time-scale Decomposition

Online Detection of Time-Variant Oscillations Based on Improved ITD

Diagnostic Checking for the Adequacy of Nonlinear Time Series Models

Variants of Non-Negative Least-Mean-Square Algorithm and Convergence Analysis

Online Statistical Inference for Nonlinear Stochastic Approximation with Markovian Data

A New Parametric Adaptive Nonstationarity Detector and Application

Nonparametric inference of stochastic differential equations based on the relative entropy rate

Finite sample inference in nonlinear regression estimation

Using the modified sample entropy to detect determinism

Inference in High-Dimensional Linear Measurement Error Models

Diagnostic Tests for Non-causal Time Series with Infinite Variance

Partial Causal Detectability of Linear Descriptor Systems and Existence of Functional ODE Estimators

A New Method to Recognize Determinism in Time Series

A New Method of Detecting Nonlinear for Time Series Based on KS Test

A Nonparametric Approach to Signal Detection in Non-Gaussian Noise

Identifying Nonlinear Dynamics with High Confidence from Sparse Data

Opportunistic Detection under a Fixed-Sample-Size Setting

Inference for High‐dimensional Linear Models with Locally Stationary Error Processes