Abstract:We consider the situation when a learner faces a set of unknown discrete distributions $(p_k)_{k\in \mathcal K}$ defined over a common alphabet $\mathcal X$, and can build for each distribution $p_k$ an individual high-probability confidence set thanks to $n_k$ observations sampled from $p_k$. The set $(p_k)_{k\in \mathcal K}$ is structured: each distribution $p_k$ is obtained from the same common, but unknown, distribution q via applying an unknown permutation to $\mathcal X$. We call this \emph{permutation-equivalence}. The goal is to build refined confidence sets \emph{exploiting} this structural property. Like other popular notions of structure (Lipschitz smoothness, Linearity, etc.) permutation-equivalence naturally appears in machine learning problems, and to benefit from its potential gain calls for a specific approach. We present a strategy to effectively exploit permutation-equivalence, and provide a finite-time high-probability bound on the size of the refined confidence sets output by the strategy. Since a refinement is not possible for too few observations in general, under mild technical assumptions, our finite-time analysis establish when the number of observations $(n_k)_{k\in \mathcal K}$ are large enough so that the output confidence sets improve over initial individual sets. We carefully characterize this event and the corresponding improvement. Further, our result implies that the size of confidence sets shrink at asymptotic rates of $O(1/\sqrt{\sum_{k\in \mathcal K} n_k})$ and $O(1/\max_{k\in K} n_{k})$, respectively for elements inside and outside the support of q, when the size of each individual confidence set shrinks at respective rates of $O(1/\sqrt{n_k})$ and $O(1/n_k)$. We illustrate the practical benefit of exploiting permutation equivalence on a reinforcement learning task.

Confidence Interval for Parameter of Exponential Distribution from Incomplete Observational Data

Empirical Likelihood Based Confidence Intervals For The Tail Index When Gamma <-1/2

Empirical Likelihood Based Inference for Conditional Pareto-type Tail Index

Asymptotic Theory for Explosive Fractional Ornstein-Uhlenbeck Processes

Confidence Intervals for Parameters of Unobserved Events

Detection of Outliers from the Two-Parameter Exponential Distribution

Confidence regions for the multinomial parameter with small sample size

On Finding the Upper Confidence Limit for a Binomial Proportion when Zero Successes are Observed

Interval Estimation for the Two-Parameter Exponential Distribution Based on Upper Record Value Data Using Bayesian Approaches

Differential geometry of fibre bundles over foliated manifolds

Binomial Distributed Data Confidence Interval Calculation: Formulas, Algorithms and Examples

Overlapping Batch Confidence Intervals on Statistical Functionals Constructed from Time Series: Application to Quantiles, Optimization, and Estimation

Interval estimation of the overlapping coefficients in an exponential family of distributions based on upper record values

Explicit Formula for Constructing Binomial Confidence Interval with Guaranteed Coverage Probability

Parameter Estimation and Hypothesis Testing of Two Pareto Distribution with Missing Data

A simple approach to construct confidence bands for a regression function with incomplete data

Estimation of fixed-accuracy confidence interval of the stress–strength reliability for inverse Pareto distribution using two-stage sampling technique

Asymptotic Properties for the Parameter Estimation in Ornstein-Uhlenbeck Process with Discrete Observations

Empirical Likelihood-Based Confidence Intervals for Data with Possible Zero Observations

How to Shrink Confidence Sets for Many Equivalent Discrete Distributions?