Abstract:Many high-dimensional practical data sets have hierarchical structures induced by graphs or time series. Such data sets are hard to process in Euclidean spaces and one often seeks low-dimensional embeddings in other space forms to perform the required learning tasks. For hierarchical data, the space of choice is a hyperbolic space because it guarantees low-distortion embeddings for tree-like structures. The geometry of hyperbolic spaces has properties not encountered in Euclidean spaces that pose challenges when trying to rigorously analyze algorithmic solutions. We propose a unified framework for learning scalable and simple hyperbolic linear classifiers with provable performance guarantees. The gist of our approach is to focus on Poincaré ball models and formulate the classification problems using tangent space formalisms. Our results include a new hyperbolic perceptron algorithm as well as an efficient and highly accurate convex optimization setup for hyperbolic support vector machine classifiers. Furthermore, we adapt our approach to accommodate second-order perceptrons, where data is preprocessed based on second-order information (correlation) to accelerate convergence, and strategic perceptrons, where potentially manipulated data arrives in an online manner and decisions are made sequentially. The excellent performance of the Poincaré second-order and strategic perceptrons shows that the proposed framework can be extended to general machine learning problems in hyperbolic spaces. Our experimental results, pertaining to synthetic, single-cell RNA-seq expression measurements, CIFAR10, Fashion-MNIST and mini-ImageNet, establish that all algorithms provably converge and have complexity comparable to those of their Euclidean counterparts. Accompanying codes can be found at: <a class="link-external link-https" href="https://github.com/thupchnsky/PoincareLinearClassification" rel="external noopener nofollow">this https URL</a>.

Geometrical Complexity of Classification Problems

Visual Complexity of Shapes: a Hierarchical Perceptual Learning Model

Geometrical probability covering algorithm

Topological learning in multiclass data sets

A Margin-based Multiclass Generalization Bound via Geometric Complexity

On Characterizing the Evolution of Embedding Space of Neural Networks using Algebraic Topology

Complexity Measures and Concept Learning

Classification with many classes: challenges and pluses

Topological Learning in Multi-Class Data Sets

Provably Accurate and Scalable Linear Classifiers in Hyperbolic Spaces

Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structures

On Geometric Structure of Activation Spaces in Neural Networks

The Impact of Geometric Complexity on Neural Collapse in Transfer Learning

Geometric Analysis of Unconstrained Feature Models with $d=K$

Topological complexity of enumerative problems and classifying spaces of $PU_n$

Why neural networks find simple solutions: the many regularizers of geometric complexity

Structure of Classifier Boundaries: Case Study for a Naive Bayes Classifier

Robust dynamics, invariant structures and topological classification

High dimensional gaussian classification

Exact Count of Boundary Pieces of ReLU Classifiers: Towards the Proper Complexity Measure for Classification

Image Characterization and Classification by Physical Complexity