Abstract:We observe n pairs of independent (but not necessarily i.i.d.) random variables X1=(W1,Y1),...,Xn=(Wn,Yn) and tackle the problem of estimating the conditional distributions Qi⋆(wi) of Yi given Wi=wi for all i∈{1,...,n} . Even though these might not be true, we base our estimator on the assumptions that the data are i.i.d. and the conditional distributions of Yi given Wi=wi belong to a one parameter exponential family Q ̄ with parameter space given by an interval I . More precisely, we pretend that these conditional distributions take the form Qθ(wi)∈Q ̄ for some θ that belongs to a VC-class Θ ̄ of functions with values in I . For each i∈{1,...,n} , we estimate Qi⋆(wi) by a distribution of the same form, i.e. Qθ̂(wi)∈Q ̄ , where θ̂=θ̂(X1,...,Xn) is a well-chosen estimator with values in Θ ̄ . We establish non-asymptotic exponential inequalities for the upper deviations of a Hellinger-type distance between the true conditional distributions of the data and the estimated one based on the exponential family Q ̄ and the class of functions Θ ̄ we chose. We show that our estimation strategy is robust to model misspecification, contamination and the presence of outliers. Besides, when the data are truly i.i.d., the exponential family Q ̄ is suitably parametrized and the conditional distributions Qi⋆(wi) of the form Qθ⋆(wi)∈Q ̄ for some unknown Hölderian function θ⋆ with values in I , we prove that the estimator θ̂ of θ⋆ is minimax (up to a logarithmic factor). Finally, we provide an algorithm for calculating θ̂ when Θ ̄ is a VC-class of functions of low or moderate dimension and we carry out a simulation study to compare its performance to that of the MLE and median-based estimators. The proof of our main result relies on an upper bound, with explicit numerical constants, on the expectation of the supremum of an empirical process over a VC-subgraph class. This bound can be of independent interest.

On improved estimation in a conditionally Gaussian regression

Improved estimation in a non-Gaussian parametric regression

A Non-Parametric Shrinkage Mean Estimation for Arbitrary Quadratic Loss Functions and Unknown Covariance Matrices

Adaptive model selection method for a conditionally Gaussian semimartingale regression in continuous time

Improved Gaussian Mean Matrix Estimators In High-Dimensional Data

Non-parametric Shrinkage Mean Estimation for Quadratic Loss Functions with Unknown Covariance Matrices

Estimating Stochastic Linear Combination of Non-Linear Regressions

Regression and conditional regression quantiles

Conditioning of Banach Space Valued Gaussian Random Variables: An Approximation Approach Based on Martingales

A Note on Improved Multivariate Normal Mean Estimation With Unknown Covariance When p Is Greater Than n

High-Dimensional Conditional Covariance Matrices Estimation Using a Factor-GARCH Model

Estimating Stochastic Linear Combination of Non-Linear Regressions Efficiently and Scalably

Covariance estimation for multivariate conditionally Gaussian dynamic linear models

Improvement of the cross-entropy method in high dimension for failure probability estimation through a one-dimensional projection without gradient estimation

Conditional Density Estimation via Weighted Logistic Regressions

Adaptive estimation of a quadratic functional by model selection

Maximum likelihood estimation for Gaussian processes under inequality constraints

Robust estimation of a regression function in exponential families

Estimating the Mean of a Non-Linear Function of Conditional Expectation

Improved parameter estimation for a family of exponential distributions

Parameter Estimation For Multivariate Generalized Gaussian Distributions