Big Outliers Versus Heavy Tails: what to use?

Lev B. Klebanov

DOI: https://doi.org/10.48550/arXiv.1611.05410

2016-11-16

Statistics Theory

Abstract:A possibility to give strong mathematical definitions of outliers and heavy tailed distributions or their modification is discussed. Some alternatives for the notion of tail index are proposed. Key words: outliers, heavy tails, tail index.

What problem does this paper attempt to address?

Heavy-Tailed Processes for Selective Shrinkage

Fabian L. Wauthier,Michael I. Jordan

DOI: https://doi.org/10.48550/arXiv.1006.3901

2010-06-23

Abstract:Heavy-tailed distributions are frequently used to enhance the robustness of regression and classification methods to outliers in output space. Often, however, we are confronted with "outliers" in input space, which are isolated observations in sparsely populated regions. We show that heavy-tailed stochastic processes (which we construct from Gaussian processes via a copula), can be used to improve robustness of regression and classification estimators to such outliers by selectively shrinking them more strongly in sparse regions than in dense regions. We carry out a theoretical analysis to show that selective shrinkage occurs, provided the marginals of the heavy-tailed process have sufficiently heavy tails. The analysis is complemented by experiments on biological data which indicate significant improvements of estimates in sparse regions while producing competitive results in dense regions.

Machine Learning,Applications,Methodology
Heavy tailed time series with extremal independence

Rafal Kulik,Philippe Soulier

DOI: https://doi.org/10.48550/arXiv.1307.1501

2014-10-09

Abstract:We consider strictly stationary heavy tailed time series whose finite-dimensional exponent measures are concentrated on axes, and hence their extremal properties cannot be tackled using classical multivariate regular variation that is suitable for time series with extremal dependence. We recover relevant information about limiting behavior of time series with extremal independence by introducing a sequence of scaling functions and conditional scaling exponent. Both quantities provide more information about joint extremes than a widely used tail dependence coefficient. We calculate the scaling functions and the scaling exponent for variety of models, including Markov chains, exponential autoregressive model, stochastic volatility with heavy tailed innovations or volatility.

Statistics Theory,Risk Management
Statistical Consequences of Fat Tails: Real World Preasymptotics, Epistemology, and Applications

Nassim Nicholas Taleb

DOI: https://doi.org/10.48550/arXiv.2001.10488

2022-11-15

Abstract:The monograph investigates the misapplication of conventional statistical techniques to fat tailed distributions and looks for remedies, when possible. Switching from thin tailed to fat tailed distributions requires more than "changing the color of the dress". Traditional asymptotics deal mainly with either n=1 or $n=\infty$, and the real world is in between, under of the "laws of the medium numbers" --which vary widely across specific distributions. Both the law of large numbers and the generalized central limit mechanisms operate in highly idiosyncratic ways outside the standard Gaussian or Levy-Stable basins of convergence. A few examples: + The sample mean is rarely in line with the population mean, with effect on "naive empiricism", but can be sometimes be estimated via parametric methods. + The "empirical distribution" is rarely empirical. + Parameter uncertainty has compounding effects on statistical metrics. + Dimension reduction (principal components) fails. + Inequality estimators (GINI or quantile contributions) are not additive and produce wrong results. + Many "biases" found in psychology become entirely rational under more sophisticated probability distributions + Most of the failures of financial economics, econometrics, and behavioral economics can be attributed to using the wrong distributions. This book, the first volume of the Technical Incerto, weaves a narrative around published journal articles.

Other Statistics,Risk Management,Applications,Methodology
Heavy-tailed probability distributions in social sciences

Lev B. Klebanov,Yulia V. Kuvaeva

DOI: https://doi.org/10.48550/arXiv.2301.09393

2023-01-23

Abstract:We present an overview of possible reasons for the appearance of heavy-tailed distributions in applications to the natural sciences. These distributions include the laws of Pareto, Lotka, and some new ones. The reasons are illustrated using suitable toy models. Keywords: heavy-tailed distributions; Pareto law; Lotka law; Zipf law; probability generating function.

Physics and Society,Probability
Fitting tails affected by truncation

Jan Beirlant,Isabel Fraga Alves,Tom Reynkens

DOI: https://doi.org/10.1214/17-EJS1286

2017-05-05

Abstract:In several applications, ultimately at the largest data, truncation effects can be observed when analysing tail characteristics of statistical distributions. In some cases truncation effects are forecasted through physical models such as the Gutenberg-Richter relation in geophysics, while at other instances the nature of the measurement process itself may cause under recovery of large values, for instance due to flooding in river discharge readings. Recently Beirlant et al. (2016) discussed tail fitting for truncated Pareto-type distributions. Using examples from earthquake analysis, hydrology and diamond valuation we demonstrate the need for a unified treatment of extreme value analysis for truncated heavy and light tails. We generalise the classical Peaks over Threshold approach for the different max-domains of attraction with shape parameter $\xi>-1/2$ to allow for truncation effects. We use a pseudo-maximum likelihood approach to estimate the model parameters and consider extreme quantile estimation and reconstruction of quantile levels before truncation whenever appropriate. We report on some simulation experiments and provide some basic asymptotic results.

Methodology
Divergence based robust estimation of the tail index through an exponential regression model

Abhik Ghosh

DOI: https://doi.org/10.1007/s10260-016-0364-9

2016-07-18

Abstract:The extreme value theory is very popular in applied sciences including finance, economics, hydrology and many other disciplines. In univariate extreme value theory, we model the data by a suitable distribution from the general max-domain of attraction characterized by its tail index; there are three broad classes of tails—the Pareto type, the Weibull type and the Gumbel type. The simplest and most common estimator of the tail index is the Hill estimator that works only for Pareto type tails and has a high bias; it is also highly non-robust in presence of outliers with respect to the assumed model. There have been some recent attempts to produce asymptotically unbiased or robust alternative to the Hill estimator; however all the robust alternatives work for any one type of tail. This paper proposes a new general estimator of the tail index that is both robust and has smaller bias under all the three tail types compared to the existing robust estimators. This essentially produces a robust generalization of the estimator proposed by Matthys and Beirlant (Stat Sin 13:853–880, 2003) under the same model approximation through a suitable exponential regression framework using the density power divergence. The robustness properties of the estimator are derived in the paper along with an extensive simulation study. A method for bias correction is also proposed with application to some real data examples.

statistics & probability
Heavy-tailed Contamination is Easier than Adversarial Contamination

Yeshwanth Cherapanamjeri,Daniel Lee

2024-11-23

Abstract:A large body of work in the statistics and computer science communities dating back to Huber (Huber, 1960) has led to statistically and computationally efficient outlier-robust estimators. Two particular outlier models have received significant attention: the adversarial and heavy-tailed models. While the former models outliers as the result of a malicious adversary manipulating the data, the latter relaxes distributional assumptions on the data allowing outliers to naturally occur as part of the data generating process. In the first setting, the goal is to develop estimators robust to the largest fraction of outliers while in the second, one seeks estimators to combat the loss of statistical efficiency, where the dependence on the failure probability is paramount. Despite these distinct motivations, the algorithmic approaches to both these settings have converged, prompting questions on the relationship between the models. In this paper, we investigate and provide a principled explanation for this phenomenon. First, we prove that any adversarially robust estimator is also resilient to heavy-tailed outliers for any statistical estimation problem with i.i.d data. As a corollary, optimal adversarially robust estimators for mean estimation, linear regression, and covariance estimation are also optimal heavy-tailed estimators. Conversely, for arguably the simplest high-dimensional estimation task of mean estimation, we construct heavy-tailed estimators whose application to the adversarial setting requires any black-box reduction to remove almost all the outliers in the data. Taken together, our results imply that heavy-tailed estimation is likely easier than adversarially robust estimation opening the door to novel algorithmic approaches for the heavy-tailed setting. Additionally, confidence intervals obtained for adversarially robust estimation also hold with high-probability.

Statistics Theory,Data Structures and Algorithms,Machine Learning,Methodology
On a Novel Skewed Generalized T Distribution: Properties, Estimations and Its Applications

Chengdi Lian,Yaohua Rong,Weihu Cheng

DOI: https://doi.org/10.1080/03610926.2024.2313034

2024-01-01

Abstract:With the progress of information technology, large amounts of asymmetric, leptokurtic, and heavy-tailed data are arising in various fields, such as finance, engineering, genetics, and medicine. It is very challenging to model those kinds of data, especially for extremely skewed data, accompanied by very high kurtosis or heavy tails. In this article, we propose a class of novel skewed generalized t distribution (SkeGTD) as a scale mixture of skewed generalized normal. The proposed SkeGTD has excellent adaptiveness to various data, because of its capability of allowing for a large range of skewness and kurtosis and its compatibility of the separated location, scale, skewness, and shape parameters. We investigate some important properties of this family of distributions. The maximum likelihood estimation, L-moments estimation, and two-step estimation for the SkeGTD are explored. To illustrate the usefulness of the proposed methodology, we present simulation studies and analyze two real datasets.
Heavy-tails in economic data: fundamental assumptions, modelling and analysis

João P. da Cruz,Pedro G. Lind

DOI: https://doi.org/10.48550/arXiv.1202.0142

2012-02-01

Computational Finance

Abstract:The study of heavy-tailed distributions in economic and financial systems has been widely addressed since financial time series has become a research subject.After the eighties, several "highly improbable" market drops were observed (e.g. the 1987 stock market drop known as "Black Monday" and on even more recent ones, already in the 21st century) that produce heavy losses that were unexplainable in a GN environment. The losses incurred in these large market drop events did not change significantly the market practices or the way regulation is done but drove some attention back to the study of heavy-tails and their underlying mechanisms. Some recent findings in these context is the scope of this manuscript.
Tempered distributions: does universal tempering procedure exist?

Lev B. Klebanov,Lenka Slámová

DOI: https://doi.org/10.48550/arXiv.1505.02068

2015-05-08

Abstract:Since the turn of the century, there has been increased interest in the application of heavy-tailed distributions, particularly stable distributions, to problems in physics and finance. Although, the tails of stable distributions provide a better fit to real-world data, they are too fat to describe empirical distributions. To remedy this drawback of stable distributions, so-called tempered variants of stable distributions have been proposed. In this paper, we argue that the tempering should be connected to the model leading to heavy tailed distribution and propose several tempering procedures in connection with corresponding models

Probability
Financial Risk and Heavy Tails

Brendan O. Bradley,Murad S. Taqqu

DOI: https://doi.org/10.1016/b978-044450896-6.50004-2

2003-01-01

Abstract:It is of great importance for those in charge of managing risk to understand how financial asset returns are distributed. Practitioners often assume for convenience that the distribu¬tion is normal. Since the 1960s, however, empirical evidence has led many to reject this assumption in favor of various heavy-tailed alternatives. In a heavy-tailed distribution the likelihood that one encounters significant deviations from the mean is much greater than in the case of the normal distribution. It is now commonly accepted that financial asset returns are, in fact, heavy-tailed. The goal of this survey is to examine how these heavy tails affect several aspects of financial portfolio theory and risk management. We describe some of the methods that one can use to deal with heavy tails and we illustrate them using the NASDAQ composite index.
Where Do Thin Tails Come From?

Nassim Nicholas Taleb

DOI: https://doi.org/10.48550/arXiv.1307.6695

2013-07-25

Risk Management

Abstract:The literature of heavy tails (typically) starts with a random walk and finds mechanisms that lead to fat tails under aggregation. We follow the inverse route and show how starting with fat tails we get to thin-tails when deriving the probability distribution of the response to a random variable. We introduce a general dose-response curve and argue that the left and right-boundedness or saturation of the response in natural things leads to thin-tails, even when the "underlying" random variable at the source of the exposure is fat-tailed.
Some notes on moment inequalities for heavy-tailed distributions

Paul Buterus,Holger Sambale

2024-10-29

Abstract:We investigate the relation between moments and tails of heavy-tailed (in particular, Pareto-type) distributions. We also discuss the sharpness of our results in a number of examples under certain regularity conditions like log-convexity. Moreover, we derive concentration bounds for polynomial chaos of any order $d$.

Probability
Using Outliers for Theory Building

Michael Gibbert,Lakshmi Balachandran Nair,Matthias Weiss,Martin Hoegl

DOI: https://doi.org/10.1177/1094428119898877

IF: 8.247

2020-01-10

Organizational Research Methods

Abstract:Outliers are promising candidates for theory building because they defy expected cause-and-effect relationships. Nonetheless, researchers often treat them as a nuisance and exclude them from further study. In fact, our analysis founds only two article using outliers for theory development in all quantitative articles published from 1993 to 2012 in six major management journals, and less than 5% cared to even mention them (relaying reasons for deleting them, mostly). To rectify this, we provide a roadmap for empirical researchers interested in theory building.

management,psychology, applied
Heavy-Tailed Distributions and Their Applications

C Su,QH Tang

DOI: https://doi.org/10.1142/9789812702715_0014

2004-01-01

Abstract:Based on some recent investigations by the authors, this paper gives a short review on the concept of heavy-tailedness. Some commonly used and newly introduced classes of heavy-tailed distributions are examined. Applications to precise large deviations and to finite time ruin probabilities are proposed.
Characterizations on Heaviness of Distribution Tails of Non-negative Variables

苏淳,胡治水,唐启鹤

DOI: https://doi.org/10.3969/j.issn.1000-0917.2003.05.012

IF: 1.675

2003-01-01

Advances in Mathematics

Abstract:In order to meet the needs of recent research in applied probability, in this paper the concepts of heavily heavy-tailed distribution and lightly heavy-tailed distribution are introduced. As main results of this paper, some criteria to distinguish the two kinds of heavy-tailed distributions are given. This paper also discusses a series of problems related to the heavily heavy-tailed distributions, and points out some useful properties of the class D.
\Ell_1-Regression with Heavy-tailed Distributions.

Lijun Zhang,Zhi-Hua Zhou

2018-01-01

Abstract:In this paper, we consider the problem of linear regression with heavy-tailed distributions. Different from previous studies that use the squared loss to measure the performance, we choose the absolute loss, which is capable of estimating the conditional median. To address the challenge that both the input and output could be heavy-tailed, we propose a truncated minimization problem, and demonstrate that it enjoys an (O) over tilde(root d/n) excess risk, where d is the dimensionality and n is the number of samples. Compared with traditional work on l(1)-regression, the main advantage of our result is that we achieve a high-probability risk bound without exponential moment conditions on the input and output. Furthermore, if the input is bounded, we show that the classical empirical risk minimization is competent for l(1)-regression even when the output is heavy-tailed.
A new approach in two-dimensional heavy-tailed distributions

Dimitrios G. Konstantinides,Charalampos D. Passalidis

2024-08-01

Abstract:We consider a new approach in the definition of two-dimensional heavy-tailed distributions. Namely, we introduce the classes of two-dimensional long-tailed, of twodimensional dominatedly varying and of two-dimensional consistently varying distributions. Next, we define the closure property with respect to two-dimensional convolution and to joint max-sum equivalence in order to study if they are satisfied by these classes. Further we examine the joint behavior of two random sums, under generalized tail asymptotic independence. Afterward we study the closure property under scalar product and two dimensional product convolution and by these results we extended our main result in the case of jointly randomly weighted sums. Our results contained some applications where we establish the asymptotic expression of the ruin probability in a two-dimensional discrete-time risk model.

Probability
How Money Got Its Tail (not Too Light; Not Too Heavy; but “just So”)

Michael R. Powers

DOI: https://doi.org/10.1108/15265940911001358

2009-01-01

The Journal of Risk Finance

Abstract:PurposeThe purpose of this paper is to explore the theoretical basis for heavy‐tailed asset‐return distributions.Design/methodology/approachThrough a simple model of asset‐price formation, one can formulate the asset‐return random variable, ln (Pt/Pt−1), as a constant plus the natural log of a ratio of Bernoulli proportions. This random variable admits of different approximations, whose distributions may be studied analytically.FindingsThe paper finds that for two reasonable approximations to the asset‐return random variable, the tails are approximately exponential. This suggests that the Gaussian assumption provides a poor “starting point” for asset‐pricing models, and empirically validated heavy‐tailed behavior is likely the result of time‐dependent components in the tail parameters.Originality/valueThe editorial offers a theoretical analysis of asset‐return distributions using parsimonious modeling assumptions.
A new method for estimating the tail index using truncated sample sequence

F. Q. Tang,D. Han

DOI: https://doi.org/10.48550/arxiv.2209.04772

2022-01-01

Abstract: This article proposes a new method of truncated estimation to estimate the tail index $\alpha$ of the extremely heavy-tailed distribution with infinite mean or variance. We not only present two truncated estimators $\hat{\alpha}$ and $\hat{\alpha}^{\prime}$ for estimating $\alpha$ ($0<\alpha \leq 1$) and $\alpha$ ($1<\alpha \leq 2$) respectively, but also prove their asymptotic statistical properties. The numerical simulation results comparing the six known estimators in estimating error, the Type I Error and the power of estimator show that the performance of the two new truncated estimators is quite good on the whole.

Big Outliers Versus Heavy Tails: what to use?

Heavy-Tailed Processes for Selective Shrinkage

Heavy tailed time series with extremal independence

Statistical Consequences of Fat Tails: Real World Preasymptotics, Epistemology, and Applications

Heavy-tailed probability distributions in social sciences

Fitting tails affected by truncation

Divergence based robust estimation of the tail index through an exponential regression model

Heavy-tailed Contamination is Easier than Adversarial Contamination

On a Novel Skewed Generalized T Distribution: Properties, Estimations and Its Applications

Heavy-tails in economic data: fundamental assumptions, modelling and analysis

Tempered distributions: does universal tempering procedure exist?

Financial Risk and Heavy Tails

Where Do Thin Tails Come From?

Some notes on moment inequalities for heavy-tailed distributions

Using Outliers for Theory Building

Heavy-Tailed Distributions and Their Applications

Characterizations on Heaviness of Distribution Tails of Non-negative Variables

\Ell_1-Regression with Heavy-tailed Distributions.

A new approach in two-dimensional heavy-tailed distributions

How Money Got Its Tail (not Too Light; Not Too Heavy; but “just So”)

A new method for estimating the tail index using truncated sample sequence