Abstract:Many high-dimensional practical data sets have hierarchical structures induced by graphs or time series. Such data sets are hard to process in Euclidean spaces and one often seeks low-dimensional embeddings in other space forms to perform the required learning tasks. For hierarchical data, the space of choice is a hyperbolic space because it guarantees low-distortion embeddings for tree-like structures. The geometry of hyperbolic spaces has properties not encountered in Euclidean spaces that pose challenges when trying to rigorously analyze algorithmic solutions. We propose a unified framework for learning scalable and simple hyperbolic linear classifiers with provable performance guarantees. The gist of our approach is to focus on Poincaré ball models and formulate the classification problems using tangent space formalisms. Our results include a new hyperbolic perceptron algorithm as well as an efficient and highly accurate convex optimization setup for hyperbolic support vector machine classifiers. Furthermore, we adapt our approach to accommodate second-order perceptrons, where data is preprocessed based on second-order information (correlation) to accelerate convergence, and strategic perceptrons, where potentially manipulated data arrives in an online manner and decisions are made sequentially. The excellent performance of the Poincaré second-order and strategic perceptrons shows that the proposed framework can be extended to general machine learning problems in hyperbolic spaces. Our experimental results, pertaining to synthetic, single-cell RNA-seq expression measurements, CIFAR10, Fashion-MNIST and mini-ImageNet, establish that all algorithms provably converge and have complexity comparable to those of their Euclidean counterparts. Accompanying codes can be found at: <a class="link-external link-https" href="https://github.com/thupchnsky/PoincareLinearClassification" rel="external noopener nofollow">this https URL</a>.

Poincaré Fréchet Mean

Evolving Hypersurfaces by Their Mean Curvature in the Background Manifold Evolving by Ricci Flow

Dynamically Stable Poincaré Embeddings for Neural Manifolds

Mean Curvature Flow of Arbitrary Codimension in Complex Projective Spaces

Gauss–Newton Method for Convex Composite Optimizations on Riemannian Manifolds

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

A Comprehensive Definition of the Geometric Mean of Convex Bodies Based on Relations Between Their $p$-Means

Variance Inequalities for Transformed Fréchet Means in Hadamard Spaces

Huber means on Riemannian manifolds

Provably Accurate and Scalable Linear Classifiers in Hyperbolic Spaces

Uniform Consistency of Generalized Fréchet Means

Mean field approximations via log-concavity

Convergence Rates for the Generalized Fréchet Mean via the Quadruple Inequality

Differential Geometry for Model Independent Analysis of Images and Other Non-Euclidean Data: Recent Developments

Deep Fréchet Regression

Optimal Riemannian metric for Poincar{é} inequalities and how to ideally precondition Langevin dymanics

An Optimal Convergence Theorem for Mean Curvature Flow of Arbitrary Codimension in Hyperbolic Spaces

The Numerical Stability of Hyperbolic Representation Learning

An Inexact Riemannian Proximal Gradient Method

Convex generalized Fréchet means in a metric tree

Convergence and Trade-Offs in Riemannian Gradient Descent and Riemannian Proximal Point