Abstract:Bayesian classification labels observations based on given prior information, namely class-a priori and class-conditional probabilities. Bayes' risk is the minimum expected classification cost that is achieved by the Bayes' test, the optimal decision rule. When no cost incurs for correct classification and unit cost is charged for misclassification, Bayes' test reduces to the maximum a posteriori decision rule, and Bayes risk simplifies to Bayes' error, the probability of error. Since calculating this probability of error is often intractable, several techniques have been devised to bound it with closed-form formula, introducing thereby measures of similarity and divergence between distributions like the Bhattacharyya coefficient and its associated Bhattacharyya distance. The Bhattacharyya upper bound can further be tightened using the Chernoff information that relies on the notion of best error exponent. In this paper, we first express Bayes' risk using the total variation distance on scaled distributions. We then elucidate and extend the Bhattacharyya and the Chernoff upper bound mechanisms using generalized weighted means. We provide as a byproduct novel notions of statistical divergences and affinity coefficients. We illustrate our technique by deriving new upper bounds for the univariate Cauchy and the multivariate $t$-distributions, and show experimentally that those bounds are not too distant to the computationally intractable Bayes' error.

Empirical Bayes Estimation of Variance Components in Two-Way Classification Random Effects Model

Empirical Bayes Estimators of Risk Premium under Variance Related Premium Principle

Empirical Bayes Test for Two-parameter Exponential Distribution under Type-II Censored Samples

Empirical Bayes Estimates for a 2-Way Cross-Classified Additive Model

Variance Analyzed under E-Bayesian Estimation and Hierarchical Bayesian Estimation for Diagnosing Failure Rates

Estimation in Covariance Components Models

Design-Robust Two-Way-Fixed-Effects Regression For Panel Data

Estimation and Inference for Linear Models with Two-Way Fixed Effects and Sparsely Matched Data

Empirical Bayes When Estimation Precision Predicts Parameters

The Superiority of the Bayes Linear Unbiased Minimum Variance Estimator under Balanced Loss Function

Bias correction for multiple covariate analysis using empirical bayesian estimation in mixed-effects models for longitudinal data

Nonparametric estimation of multivariate copula using empirical bayes method

On estimation of mean squared errors of benchmarked empirical Bayes estimators

Efficient Empirical Bayes prediction under check loss using Asymptotic Risk Estimates

Empirical Bayes for Large-scale Randomized Experiments: a Spectral Approach

Entropy Balancing is Doubly Robust

Learning from a lot: Empirical Bayes in high-dimensional prediction settings

A Comparison of REML and Bayesian Estimation of Variance Components for Unbalanced One-way Random Effects Models With Small Sample Sizes

Generalized Bhattacharyya and Chernoff upper bounds on Bayes error using quasi-arithmetic means

A General Framework for Empirical Bayes Estimation in Discrete Linear Exponential Family

Robust Fitting of Mixture Models using Weighted Complete Estimating Equations