Halfspace depth does not characterize probability distributions

Stanislav Nagy

DOI: https://doi.org/10.1007/s00362-019-01130-x

2018-10-22

Abstract:We give examples of different multivariate probability distributions whose halfspace depths coincide at all points of the sample space.

Probability

What problem does this paper attempt to address?

Distributionally robust halfspace depth

Jevgenijs Ivanovs,Pavlo Mozharovskyi

2024-05-10

Abstract:Tukey's halfspace depth can be seen as a stochastic program and as such it is not guarded against optimizer's curse, so that a limited training sample may easily result in a poor out-of-sample performance. We propose a generalized halfspace depth concept relying on the recent advances in distributionally robust optimization, where every halfspace is examined using the respective worst-case distribution in the Wasserstein ball of radius $\delta\geq 0$ centered at the empirical law. This new depth can be seen as a smoothed and regularized classical halfspace depth which is retrieved as $\delta\downarrow 0$. It inherits most of the main properties of the latter and, additionally, enjoys various new attractive features such as continuity and strict positivity beyond the convex hull of the support. We provide numerical illustrations of the new depth and its advantages, and develop some fundamental theory. In particular, we study the upper level sets and the median region including their breakdown properties.

Optimization and Control,Methodology
Theoretical properties of angular halfspace depth

Stanislav Nagy,Petra Laketa

2024-02-13

Abstract:The angular halfspace depth (ahD) is a natural modification of the celebrated halfspace (or Tukey) depth to the setup of directional data. It allows us to define elements of nonparametric inference, such as the median, the inter-quantile regions, or the rank statistics, for datasets supported in the unit sphere. Despite being introduced in 1987, ahD has never received ample recognition in the literature, mainly due to the lack of efficient algorithms for its computation. With the recent progress on the computational front, ahD however exhibits the potential for developing viable nonparametric statistics techniques for directional datasets. In this paper, we thoroughly treat the theoretical properties of ahD. We show that similarly to the classical halfspace depth for multivariate data, also ahD satisfies many desirable properties of a statistical depth function. Further, we derive uniform continuity/consistency results for the associated set of directional medians, and the central regions of ahD, the latter representing a depth-based analogue of the quantiles for directional data.

Statistics Theory,Methodology
Halfspace depth for general measures: The ray basis theorem and its consequences

Petra Laketa,Stanislav Nagy

DOI: https://doi.org/10.1007/s00362-021-01259-8

2021-05-31

Abstract:The halfspace depth is a prominent tool of nonparametric multivariate analysis. The upper level sets of the depth, termed the trimmed regions of a measure, serve as a natural generalization of the quantiles and inter-quantile regions to higher-dimensional spaces. The smallest non-empty trimmed region, coined the halfspace median of a measure, generalizes the median. We focus on the (inverse) ray basis theorem for the halfspace depth, a crucial theoretical result that characterizes the halfspace median by a covering property. First, a novel elementary proof of that statement is provided, under minimal assumptions on the underlying measure. The proof applies not only to the median, but also to other trimmed regions. Motivated by the technical development of the amended ray basis theorem, we specify connections between the trimmed regions, floating bodies, and additional equi-affine convex sets related to the depth. As a consequence, minimal conditions for the strict monotonicity of the depth are obtained. Applications to the computation of the depth and robust estimation are outlined.

Statistics Theory,Methodology
The half-space Airy stat process

Dan Betea,Patrik Ferrari,Alessandra Occelli

2020-12-19

Abstract:We study the multipoint distribution of stationary half-space last passage percolation with exponentially weighted times. We derive both finite-size and asymptotic results for this distribution. In the latter case we observe a new one-parameter process we call half-space Airy stat. It is a one-parameter generalization of the Airy stat process of Baik-Ferrari-P\'ech\'e, which is recovered far away from the diagonal. All these results extend the one-point results previously proven by the authors.

Mathematical Physics,Probability
Half-space Mass: a Maximally Robust and Efficient Data Depth Method

Bo Chen,Kai Ming Ting,Takashi Washio,Gholamreza Haffari

DOI: https://doi.org/10.1007/s10994-015-5524-x

IF: 5.414

2015-01-01

Machine Learning

Abstract:Data depth is a statistical method which models data distribution in terms of center-outward ranking rather than density or linear ranking. While there are a lot of academic interests, its applications are hampered by the lack of a method which is both robust and efficient. This paper introduces Half-Space Mass which is a significantly improved version of half-space data depth. Half-Space Mass is the only data depth method which is both robust and efficient, as far as we know. We also reveal four theoretical properties of Half-Space Mass : (i) its resultant mass distribution is concave regardless of the underlying density distribution, (ii) its maximum point is unique which can be considered as median, (iii) the median is maximally robust, and (iv) its estimation extends to a higher dimensional space in which the convex hull of the dataset occupies zero volume. We demonstrate the power of Half-Space Mass through its applications in two tasks. In anomaly detection, being a maximally robust location estimator leads directly to a robust anomaly detector that yields a better detection accuracy than half-space depth; and it runs orders of magnitude faster than L_2 depth, an existing maximally robust location estimator. In clustering, the Half-Space Mass version of K-means overcomes three weaknesses of K-means.
Contrastive Moments: Unsupervised Halfspace Learning in Polynomial Time

Xinyuan Cao,Santosh S. Vempala

2023-11-03

Abstract:We give a polynomial-time algorithm for learning high-dimensional halfspaces with margins in $d$-dimensional space to within desired TV distance when the ambient distribution is an unknown affine transformation of the $d$-fold product of an (unknown) symmetric one-dimensional logconcave distribution, and the halfspace is introduced by deleting at least an $\epsilon$ fraction of the data in one of the component distributions. Notably, our algorithm does not need labels and establishes the unique (and efficient) identifiability of the hidden halfspace under this distributional assumption. The sample and time complexity of the algorithm are polynomial in the dimension and $1/\epsilon$. The algorithm uses only the first two moments of suitable re-weightings of the empirical distribution, which we call contrastive moments; its analysis uses classical facts about generalized Dirichlet polynomials and relies crucially on a new monotonicity property of the moment ratio of truncations of logconcave distributions. Such algorithms, based only on first and second moments were suggested in earlier work, but hitherto eluded rigorous guarantees. Prior work addressed the special case when the underlying distribution is Gaussian via Non-Gaussian Component Analysis. We improve on this by providing polytime guarantees based on Total Variation (TV) distance, in place of existing moment-bound guarantees that can be super-polynomial. Our work is also the first to go beyond Gaussians in this setting.

Machine Learning,Probability
High Dimensional Space Oddity

Haim Bar,Vladimir Pozdnyakov

2024-09-20

Abstract:In his 1996 paper, Talagrand highlighted that the Law of Large Numbers (LLN) for independent random variables can be viewed as a geometric property of multidimensional product spaces. This phenomenon is known as the concentration of measure. To illustrate this profound connection between geometry and probability theory, we consider a seemingly intractable geometric problem in multidimensional Euclidean space and solve it using standard probabilistic tools such as the LLN and the Central Limit Theorem (CLT).

Probability,Statistics Theory
Fast kernel half-space depth for data with non-convex supports

Arturo Castellanos,Pavlo Mozharovskyi,Florence d'Alché-Buc,Hicham Janati

2023-12-22

Abstract:Data depth is a statistical function that generalizes order and quantiles to the multivariate setting and beyond, with applications spanning over descriptive and visual statistics, anomaly detection, testing, etc. The celebrated halfspace depth exploits data geometry via an optimization program to deliver properties of invariances, robustness, and non-parametricity. Nevertheless, it implicitly assumes convex data supports and requires exponential computational cost. To tackle distribution's multimodality, we extend the halfspace depth in a Reproducing Kernel Hilbert Space (RKHS). We show that the obtained depth is intuitive and establish its consistency with provable concentration bounds that allow for homogeneity testing. The proposed depth can be computed using manifold gradient making faster than halfspace depth by several orders of magnitude. The performance of our depth is demonstrated through numerical simulations as well as applications such as anomaly detection on real data and homogeneity testing.

Machine Learning
Regularized Halfspace Depth for Functional Data

Hyemin Yeon,Xiongtao Dai,Sara Lopez-Pintado

2024-05-24

Abstract:Data depth is a powerful nonparametric tool originally proposed to rank multivariate data from center outward. In this context, one of the most archetypical depth notions is Tukey's halfspace depth. In the last few decades notions of depth have also been proposed for functional data. However, Tukey's depth cannot be extended to handle functional data because of its degeneracy. Here, we propose a new halfspace depth for functional data which avoids degeneracy by regularization. The halfspace projection directions are constrained to have a small reproducing kernel Hilbert space norm. Desirable theoretical properties of the proposed depth, such as isometry invariance, maximality at center, monotonicity relative to a deepest point, upper semi-continuity, and consistency are established. Moreover, the regularized halfspace depth can rank functional data with varying emphasis in shape or magnitude, depending on the regularization. A new outlier detection approach is also proposed, which is capable of detecting both shape and magnitude outliers. It is applicable to trajectories in $L^2$, a very general space of functions that include non-smooth trajectories. Based on extensive numerical studies, our methods are shown to perform well in terms of detecting outliers of different types. Three real data examples showcase the proposed depth notion.

Methodology,Statistics Theory
Negative Probability

Nick Polson,Vadim Sokolov

2024-09-05

Abstract:Negative probabilities arise primarily in physics, statistical quantum mechanics and quantum computing. Negative probabilities arise as mixing distributions of unobserved latent variables in Bayesian modeling. Our goal is to provide a link between these two viewpoints. Bartlett provides a definition of negative probabilities based on extraordinary random variables and properties of their characteristic function. A version of Bayes rule is given with negative mixing weights. The classic half coin distribution and Polya-Gamma mixing is discussed. Heisenberg's principle of uncertainty and the duality of scale mixtures of Normals is also discussed. A number of examples of dual densities with negative mixing measures are provided including the Linnik and Wigner distributions. Finally, we conclude with directions for future research.

Quantum Physics,Statistics Theory
Depth Patterns

Annika Betken,Alexander Schnurr

2024-01-24

Abstract:We establish a definition of ordinal patterns for multivariate time series data based on the concept of Tukey's halfspace depth. Given the definition of these \emph{depth patterns}, we are interested in the probabilities of observing specific patterns in a time series. For this, we consider the relative frequency of depth patterns as natural estimators for their occurrence probabilities. Depending on the choice of reference distribution and the relation between reference and data distribution, we distinguish different settings that are considered separately. Within these settings we study statistical properties of ordinal pattern probabilities, establishing consistency and asymptotic normality under the assumption of weakly dependent time series data. Since our concept only depends on ordinal depth information, the resulting values are robust under small perturbations and measurement errors.

Statistics Theory,Probability
PDE characterisation of geometric distribution functions and quantiles

Dimitri Konen

2024-04-10

Abstract:We show that in any Euclidean space, an arbitrary probability measure can be reconstructed explicitly by its geometric (or spatial) distribution function. The reconstruction takes the form of a (potentially fractional) linear PDE, where the differential operator is given in closed form. This result implies that, contrary to a common belief in the statistical depth community, geometric cdf's in principle provide exact control over the probability content of all depth regions. We present a comprehensive study of the regularity of the geometric cdf, and show that a continuous density in general does not give rise to a geometric cdf with enough regularity to reconstruct the density pointwise. Surprisingly, we prove that the reconstruction displays different behaviours in odd and even dimension: it is local in odd dimension and completely nonlocal in even dimension. We investigate this issue and provide a partial counterpart for even dimensions, and establish a general representation formula of the geometric cdf of spherically symmetric probability laws in odd dimensions. We provide explicit examples of the reconstruction of a density from its geometric cdf in dimension 2 and 3.

Mathematics
Shift invariance of half space integrable models

Jimmy He

DOI: https://doi.org/10.48550/arXiv.2205.13029

2022-05-26

Abstract:We formulate and establish symmetries of certain integrable half space models, analogous to recent results on symmetries for models in a full space. Our starting point is the colored stochastic six vertex model in a half space, from which we obtain results on the asymmetric simple exclusion process, as well as for the beta polymer through a fusion procedure which may be of independent interest. As an application, we establish a distributional identity between the absorption time in a type $B$ analogue of the oriented swap process and last passage times in a half space, establishing the Baik--Ben Arous--Péché phase transition for the absorption time. The proof uses Hecke algebras and integrability of the six vertex model through the Yang--Baxter and reflection equations.

Probability,Mathematical Physics,Combinatorics
Illumination depth

Stanislav Nagy,Jiří Dvořák

DOI: https://doi.org/10.1080/10618600.2020.1776717

2019-05-10

Abstract:The concept of illumination bodies studied in convex geometry is used to amend the halfspace depth for multivariate data. The proposed notion of illumination enables finer resolution of the sample points, naturally breaks ties in the associated depth-based ordering, and introduces a depth-like function for points outside the convex hull of the support of the probability measure. The illumination is, in a certain sense, dual to the halfspace depth mapping, and shares the majority of its beneficial properties. It is affine invariant, robust, uniformly consistent, and aligns well with common probability distributions.

Statistics Theory
Generalization of the simplicial depth: no vanishment outside the convex hull of the distribution support

Giacomo Francisci,Alicia Nieto-Reyes,Claudio Agostinelli

2024-01-12

Abstract:The simplicial depth, like other relevant multivariate statistical data depth functions, vanishes right outside the convex hull of the support of the distribution with respect to which the depth is computed. This is problematic when it is required to differentiate among points outside the convex hull of the distribution support, with respect to which the depth is computed, based on their depth values. We provide the first proposal for simplicial depth which do not vanish right outside the convex hull of the distribution. The properties of the proposal and of the corresponding estimator are studied theoretically and by means of Monte Carlo simulations and analysis of datasets.

Statistics Theory
Reliable Learning of Halfspaces under Gaussian Marginals

Ilias Diakonikolas,Lisheng Ren,Nikos Zarifis

2024-11-18

Abstract:We study the problem of PAC learning halfspaces in the reliable agnostic model of Kalai et al. (2012). The reliable PAC model captures learning scenarios where one type of error is costlier than the others. Our main positive result is a new algorithm for reliable learning of Gaussian halfspaces on $\mathbb{R}^d$ with sample and computational complexity $$d^{O(\log (\min\{1/\alpha, 1/\epsilon\}))}\min (2^{\log(1/\epsilon)^{O(\log (1/\alpha))}},2^{\mathrm{poly}(1/\epsilon)})\;,$$ where $\epsilon$ is the excess error and $\alpha$ is the bias of the optimal halfspace. We complement our upper bound with a Statistical Query lower bound suggesting that the $d^{\Omega(\log (1/\alpha))}$ dependence is best possible. Conceptually, our results imply a strong computational separation between reliable agnostic learning and standard agnostic learning of halfspaces in the Gaussian setting.

Machine Learning,Data Structures and Algorithms
Extremal behaviour and convergence rates for sample--based geometric quantiles and half space depths

Sibsankar Singha,Marie Kratz,Sreekar Vadlamani

2024-02-04

Abstract:We consider the empirical versions of geometric quantile and halfspace depth, and study their extremal behaviour as a function of the sample size. The objective of this study is to establish connection between the rates of convergence and tail behaviour of the corresponding underlying distributions. The intricate interplay between the sample size and the parameter driving the extremal behaviour forms the main result of this analysis. In the process, we also fill certain gaps in the understanding of population versions of geometric quantile and halfspace depth.

Statistics Theory
Functional limit laws for the intensity measure of point processes and applications

Giacomo Francisci,Anand N. Vidyashankar

2024-02-08

Abstract:Motivated by applications to the study of depth functions for tree-indexed random variables generated by point processes, we describe functional limit theorems for the intensity measure of point processes. Specifically, we establish uniform laws of large numbers and uniform central limit theorems over a class of bounded measurable functions for estimates of the intensity measure. Using these results, we derive the uniform asymptotic properties of half-space depth and, as corollaries, obtain the asymptotic behavior of medians and other quantiles of the standardized intensity measure. Additionally, we obtain uniform concentration upper bound for the estimator of half-space depth. As a consequence of our results, we also derive uniform consistency and uniform asymptotic normality of Lotka-Nagaev and Harris-type estimators for the Laplace transform of the point processes in a branching random walk.

Probability,Statistics Theory
Density in the half-line Schwartz space of functions whose Fourier-Laplace transform has natural boundary the real line

Andreas Chatziafratis,Telemachos Hatziafratis

2024-03-06

Abstract:In this note, we prove that the Fourier-Laplace transform of the typical function (i.e., generic in the sense of Baire category theorem) in the Schwartz class of the half-line, being analytic in the lower half of the complex plane, has natural boundary the axis of the real numbers. We also provide variations and generalizations.

Complex Variables,Mathematical Physics,Functional Analysis,General Topology
Theory of angular depth for classification of directional data

Stanislav Nagy,Houyem Demni,Davide Buttarazzi,Giovanni C. Porzio

DOI: https://doi.org/10.1007/s11634-023-00557-3

2023-09-25

Advances in Data Analysis and Classification

Abstract:Depth functions offer an array of tools that enable the introduction of quantile- and ranking-like approaches to multivariate and non-Euclidean datasets. We investigate the potential of using depths in the problem of nonparametric supervised classification of directional data, that is classification of data that naturally live on the unit sphere of a Euclidean space. In this paper, we address the problem mainly from a theoretical side, with the final goal of offering guidelines on which angular depth function should be adopted in classifying directional data. A set of desirable properties of an angular depth is put forward. With respect to these properties, we compare and contrast the most widely used angular depth functions. Simulated and real data are eventually exploited to showcase the main implications of the discussed theoretical results, with an emphasis on potentials and limits of the often disregarded angular halfspace depth.

statistics & probability

Halfspace depth does not characterize probability distributions

Distributionally robust halfspace depth

Theoretical properties of angular halfspace depth

Halfspace depth for general measures: The ray basis theorem and its consequences

The half-space Airy stat process

Half-space Mass: a Maximally Robust and Efficient Data Depth Method

Contrastive Moments: Unsupervised Halfspace Learning in Polynomial Time

High Dimensional Space Oddity

Fast kernel half-space depth for data with non-convex supports

Regularized Halfspace Depth for Functional Data

Negative Probability

Depth Patterns

PDE characterisation of geometric distribution functions and quantiles

Shift invariance of half space integrable models

Illumination depth

Generalization of the simplicial depth: no vanishment outside the convex hull of the distribution support

Reliable Learning of Halfspaces under Gaussian Marginals

Extremal behaviour and convergence rates for sample--based geometric quantiles and half space depths

Functional limit laws for the intensity measure of point processes and applications

Density in the half-line Schwartz space of functions whose Fourier-Laplace transform has natural boundary the real line

Theory of angular depth for classification of directional data