Abstract:The study of loss function distributions is critical to characterize a model's behaviour on a given machine learning problem. For example, while the quality of a model is commonly determined by the average loss assessed on a testing set, this quantity does not reflect the existence of the true mean of the loss distribution. Indeed, the finiteness of the statistical moments of the loss distribution is related to the thickness of its tails, which are generally unknown. Since typical cross-validation schemes determine a family of testing loss distributions conditioned on the training samples, the total loss distribution must be recovered by marginalizing over the space of training sets. As we show in this work, the finiteness of the sampling procedure negatively affects the reliability and efficiency of classical tail estimation methods from the Extreme Value Theory, such as the Peaks-Over-Threshold approach. In this work we tackle this issue by developing a novel general theory for estimating the tails of marginal distributions, when there exists a large variability between locations of the individual conditional distributions underlying the marginal. To this end, we demonstrate that under some regularity conditions, the shape parameter of the marginal distribution is the maximum tail shape parameter of the family of conditional distributions. We term this estimation approach as Cross Tail Estimation (CTE). We test cross-tail estimation in a series of experiments on simulated and real data, showing the improved robustness and quality of tail estimation as compared to classical approaches, and providing evidence for the relationship between overfitting and loss distribution tail thickness.

Loss modeling with many-parameter distributions

Multivariate Risk Models under Heavy-Tailed Risks

Stochastic Loss Reserving: Dependence and Estimation

Two-sided distributions with applications in insurance loss modeling

Loss as the Inconsistency of a Probabilistic Dependency Graph: Choose Your Model, Not Your Loss Function

Generalized Logistic Models and its orthant tail dependence

A Tractable Class of Multivariate Phase-Type Distributions for Loss Modeling

Model Uncertainty and Selection of Risk Models for Left-Truncated and Right-Censored Loss Data

Bayesian and minimax estimators of loss

A new class of composite GBII regression models with varying threshold for modeling heavy-tailed data

Heavy-Tailed Loss Frequencies from Mixtures of Negative Binomial and Poisson Counts

On Tail Decay Rate Estimation of Loss Function Distributions

Examples in which misspecification of a random effects distribution reduces efficiency, and possible remedies

The Impact of Loss Estimation on Gibbs Measures

An Alternative Probabilistic Interpretation of the Huber Loss

A Bayesian Skew-heavy-tailed modelling for loss reserving

A Bivariate Extension of Type-II Generalized Crack Distribution for Modeling Heavy-Tailed Losses

Analytic Loss Distributional Approach Model for Operational Risk from the alpha-Stable Doubly Stochastic Compound Processes and Implications for Capital Allocation

Parameter Estimation of Birnbaum-Saunders Distribution under Competing Risks Using the Quantile Variant of the Expectation-Maximization Algorithm

A New Family of Error Distributions for Bayesian Quantile Regression

Almost Worst Case Distributions in Multiple Priors Models