Abstract:Many recent developments in causal inference, and functional estimation problems more generally, have been motivated by the fact that classical one-step (first-order) debiasing methods, or their more recent sample-split double machine-learning avatars, can outperform plugin estimators under surprisingly weak conditions. These first-order corrections improve on plugin estimators in a black-box fashion, and consequently are often used in conjunction with powerful off-the-shelf estimation methods. These first-order methods are however provably suboptimal in a minimax sense for functional estimation when the nuisance functions live in Holder-type function spaces. This suboptimality of first-order debiasing has motivated the development of "higher-order" debiasing methods. The resulting estimators are, in some cases, provably optimal over Holder-type spaces, but both the estimators which are minimax-optimal and their analyses are crucially tied to properties of the underlying function space. In this paper we investigate the fundamental limits of structure-agnostic functional estimation, where relatively weak conditions are placed on the underlying nuisance functions. We show that there is a strong sense in which existing first-order methods are optimal. We achieve this goal by providing a formalization of the problem of functional estimation with black-box nuisance function estimates, and deriving minimax lower bounds for this problem. Our results highlight some clear tradeoffs in functional estimation -- if we wish to remain agnostic to the underlying nuisance function spaces, impose only high-level rate conditions, and maintain compatibility with black-box nuisance estimators then first-order methods are optimal. When we have an understanding of the structure of the underlying nuisance functions then carefully constructed higher-order estimators can outperform first-order estimators.

Adaptive Estimation of Nonparametric Functionals.

An adaptation theory for nonparametric confidence intervals

Statistical Limits of Adaptive Linear Models: Low-Dimensional Estimation and Inference

Central Limit Theorems of a Recursive Stochastic Algorithm with Applications to Adaptive Designs

Adaptive estimation of irregular mean and covariance functions

Adaptive estimation for Weakly Dependent Functional Times Series

Asymptotic equivalence and adaptive estimation for robust nonparametric regression

Adaptive Estimation and Uniform Confidence Bands for Nonparametric Structural Functions and Elasticities

Robust adaptive finite-time parameter estimation for linearly parameterized nonlinear systems

An Adaptive Estimation for Covariate-Adjusted Nonparametric Regression Model

Adaptive estimation of a function from its Exponential Radon Transform in presence of noise

Uniform approximation of vectors using adaptive randomized information

Adaptive Global Confidence Band for Nonparametric Regression: an Empirical Likelihood Method

The Fundamental Limits of Structure-Agnostic Functional Estimation

Adaptive Inference in Multivariate Nonparametric Regression Models Under Monotonicity

Adaptive estimation over anisotropic functional classes via oracle approach

Functional Regression with Unknown Manifold Structures

Estimation for semi-parametric varying coefficient non-linear regression function under the viewpoint of functional approximation

Benign overfitting and adaptive nonparametric regression

A Nonparametric Regression Estimator That Adapts to Error Distribution of Unknown Form

Nonparametric Functional Analysis of Generalized Linear Models Under Nonlinear Constraints