Abstract:We observe n pairs of independent (but not necessarily i.i.d.) random variables X1=(W1,Y1),...,Xn=(Wn,Yn) and tackle the problem of estimating the conditional distributions Qi⋆(wi) of Yi given Wi=wi for all i∈{1,...,n} . Even though these might not be true, we base our estimator on the assumptions that the data are i.i.d. and the conditional distributions of Yi given Wi=wi belong to a one parameter exponential family Q ̄ with parameter space given by an interval I . More precisely, we pretend that these conditional distributions take the form Qθ(wi)∈Q ̄ for some θ that belongs to a VC-class Θ ̄ of functions with values in I . For each i∈{1,...,n} , we estimate Qi⋆(wi) by a distribution of the same form, i.e. Qθ̂(wi)∈Q ̄ , where θ̂=θ̂(X1,...,Xn) is a well-chosen estimator with values in Θ ̄ . We establish non-asymptotic exponential inequalities for the upper deviations of a Hellinger-type distance between the true conditional distributions of the data and the estimated one based on the exponential family Q ̄ and the class of functions Θ ̄ we chose. We show that our estimation strategy is robust to model misspecification, contamination and the presence of outliers. Besides, when the data are truly i.i.d., the exponential family Q ̄ is suitably parametrized and the conditional distributions Qi⋆(wi) of the form Qθ⋆(wi)∈Q ̄ for some unknown Hölderian function θ⋆ with values in I , we prove that the estimator θ̂ of θ⋆ is minimax (up to a logarithmic factor). Finally, we provide an algorithm for calculating θ̂ when Θ ̄ is a VC-class of functions of low or moderate dimension and we carry out a simulation study to compare its performance to that of the MLE and median-based estimators. The proof of our main result relies on an upper bound, with explicit numerical constants, on the expectation of the supremum of an empirical process over a VC-subgraph class. This bound can be of independent interest.

Improved parameter estimation for a family of exponential distributions

Parameter Estimation in Exponential Models by Linear and Nonlinear Fitting Methods.

Improved estimation in a non-Gaussian parametric regression

Parameter Identification for Fractional Ornstein–Uhlenbeck Processes Based on Discrete Observation

Asymptotic Theory for Explosive Fractional Ornstein-Uhlenbeck Processes

Estimation in functional regression for general exponential families

Maximum likelihood estimation for a special exponential family under random double-truncation

Robust estimation of a regression function in exponential families

A new semi-parametric family of probability distributions for survival analysis

Parameters Estimation for Probability Density Functions Based on Improved Empirical Characteristic Function

ASYMPTOTICALLY OPTIMAL EMPIRICAL BAYES ESTIMATION FOR THE PARAMETERS OF MULTI-PARAMETER DISCRETE EXPONENTIAL FAMILY

On Computationally Efficient Learning of Exponential Family Distributions

A Note on the Convergence Rates for Empirical Bayes Estimators of Parameters in Multiple-Parameter Exponential Families

Parameter Estimation in Nonlinear Multivariate Stochastic Differential Equations Based on Splitting Schemes

The Improved Moment-Type Methods For The Parameters Estimation Of The Gaussian And Symmetric Alpha Stable Distributions Based On Empirical Characteristic Function

A Computationally Efficient Method for Learning Exponential Family Distributions

Nonparametric estimation of a mixing distribution for a family of linear stochastic dynamical systems

On improved estimation in a conditionally Gaussian regression

Parameter Estimation with Increased Precision for Elliptic and Hypo-elliptic Diffusions

Parameter Estimation for Multivariate Nonlinear Stochastic Differential Equation Models: A Comparison Study