Abstract:We propose a general purpose confidence interval procedure (CIP) for statistical functionals constructed using data from a stationary time series. The procedures we propose are based on derived distribution-free analogues of the $\chi^2$ and Student's $t$ random variables for the statistical functional context, and hence apply in a wide variety of settings including quantile estimation, gradient estimation, M-estimation, CVAR-estimation, and arrival process rate estimation, apart from more traditional statistical settings. Like the method of subsampling, we use overlapping batches of time series data to estimate the underlying variance parameter; unlike subsampling and the bootstrap, however, we assume that the implied point estimator of the statistical functional obeys a central limit theorem (CLT) to help identify the weak asymptotics (called OB-x limits, x=I,II,III) of batched Studentized statistics. The OB-x limits, certain functionals of the Wiener process parameterized by the size of the batches and the extent of their overlap, form the essential machinery for characterizing dependence, and consequently the correctness of the proposed CIPs. The message from extensive numerical experimentation is that in settings where a functional CLT on the point estimator is in effect, using \emph{large overlapping batches} alongside OB-x critical values yields confidence intervals that are often of significantly higher quality than those obtained from more generic methods like subsampling or the bootstrap. We illustrate using examples from CVaR estimation, ARMA parameter estimation, and NHPP rate estimation; R and MATLAB code for OB-x critical values is available at~\texttt{<a class="link-external link-http" href="http://web.ics.purdue.edu/~pasupath/" rel="external noopener nofollow">this http URL</a>}.

Efficient representations of upper limits and confidence intervals for experimental data

Functional upper limits

Importance sampling method for Feldman-Cousins confidence intervals

Another Look at Confidence Intervals: Proposal for a More Relevant and Transparent Approach

Extreme Expected Values and Their Applications in Quantum Metrology

A Geometric Approach to Confidence Regions and Bands for Functional Parameters

Design and analysis of shortest two-sided confidence intervals for a probability under prior information

Advantages of variance reduction techniques in establishing confidence intervals for quantiles

Some ways of combining optimum interval upper limits

Concentration-based confidence intervals for U-statistics

Confidence Intervals and Hypothesis Testing for High-Dimensional Regression

On Improved Confidence Bounds for System Reliability

An importance sampling method for Feldman-Cousins confidence intervals

Extreme Limit Theory of Competing Risks under Power Normalization

Overlapping Batch Confidence Intervals on Statistical Functionals Constructed from Time Series: Application to Quantiles, Optimization, and Estimation

Convergence to the Asymptotic Large Deviation Limit

High-probability minimax lower bounds

Confidence Intervals for the Probability of Superiority Effect Size Measure and the Area Under a Receiver Operating Characteristic Curve

Optimal Universal Uncertainty Relations

Probabilistic performance estimators for computational chemistry methods: the empirical cumulative distribution function of absolute errors