Abstract:The advent of large-scale inference has spurred reexamination of conventional statistical thinking. In a Gaussian model for $n$ many $z$-scores with at most $k < \frac{n}{2}$ nonnulls, Efron suggests estimating the location and scale parameters of the null distribution. Placing no assumptions on the nonnull effects, the statistical task can be viewed as a robust estimation problem. However, the best known robust estimators fail to be consistent in the regime $k \asymp n$ which is especially relevant in large-scale inference. The failure of estimators which are minimax rate-optimal with respect to other formulations of robustness (e.g. Huber's contamination model) might suggest the impossibility of consistent estimation in this regime and, consequently, a major weakness of Efron's suggestion. A sound evaluation of Efron's model thus requires a complete understanding of consistency. We sharply characterize the regime of $k$ for which consistent estimation is possible and further establish the minimax estimation rates. It is shown consistent estimation of the location parameter is possible if and only if $\frac{n}{2} - k = \omega(\sqrt{n})$, and consistent estimation of the scale parameter is possible in the entire regime $k < \frac{n}{2}$. Faster rates than those in Huber's contamination model are achievable by exploiting the Gaussian character of the data. The minimax upper bound is obtained by considering estimators based on the empirical characteristic function. The minimax lower bound involves constructing two marginal distributions whose characteristic functions match on a wide interval containing zero. The construction notably differs from those in the literature by sharply capturing a scaling of $n-2k$ in the minimax estimation rate of the location.

Inference on Consensus Ranking of Distributions

Confidence distributions and hypothesis testing

A UNIFIED STUDY OF NONPARAMETRIC INFERENCE FOR MONOTONE FUNCTIONS

Optimal estimation of the null distribution in large-scale inference

Generic Inference on Quantile and Quantile Effect Functions for Discrete Outcomes

Estimation and Inference in Distributional Reinforcement Learning

Concentration of Randomized Functions of Uniformly Bounded Variation

Ranking Inferences Based on the Top Choice of Multiway Comparisons

Beyond the Sharp Null: Randomization Inference, Bounded Null Hypotheses, and Confidence Intervals for Maximum Effects

Inference on order restricted means of inverse Gaussian populations under heteroscedasticity

A Nonparametric Test of Mth-Degree Inverse Stochastic Dominance

A Distributions-based Approach for Data-Consistent Inversion

Distributional Overlap: Simple, Multivariate, Parametric, and Nonparametric Tests for Alienation, Convergence, and General Distributional Difference Issues

Distribution-uniform anytime-valid sequential inference

Estimating the null distribution for conditional inference and genome-scale screening

Statistical divergences in high-dimensional hypothesis testing and a modern technique for estimating them

Optimal Private and Communication Constraint Distributed Goodness-of-Fit Testing for Discrete Distributions in the Large Sample Regime

Asymptotic Distribution and Simultaneous Confidence Bands for Ratios of Quantile Functions

Credible Distributions of Overall Ranking of Entities

Econometric Inference on a Large Bayesian Game with Heterogeneous Beliefs

Winners with Confidence: Discrete Argmin Inference with an Application to Model Selection