Abstract:Many high-dimensional practical data sets have hierarchical structures induced by graphs or time series. Such data sets are hard to process in Euclidean spaces and one often seeks low-dimensional embeddings in other space forms to perform the required learning tasks. For hierarchical data, the space of choice is a hyperbolic space because it guarantees low-distortion embeddings for tree-like structures. The geometry of hyperbolic spaces has properties not encountered in Euclidean spaces that pose challenges when trying to rigorously analyze algorithmic solutions. We propose a unified framework for learning scalable and simple hyperbolic linear classifiers with provable performance guarantees. The gist of our approach is to focus on Poincaré ball models and formulate the classification problems using tangent space formalisms. Our results include a new hyperbolic perceptron algorithm as well as an efficient and highly accurate convex optimization setup for hyperbolic support vector machine classifiers. Furthermore, we adapt our approach to accommodate second-order perceptrons, where data is preprocessed based on second-order information (correlation) to accelerate convergence, and strategic perceptrons, where potentially manipulated data arrives in an online manner and decisions are made sequentially. The excellent performance of the Poincaré second-order and strategic perceptrons shows that the proposed framework can be extended to general machine learning problems in hyperbolic spaces. Our experimental results, pertaining to synthetic, single-cell RNA-seq expression measurements, CIFAR10, Fashion-MNIST and mini-ImageNet, establish that all algorithms provably converge and have complexity comparable to those of their Euclidean counterparts. Accompanying codes can be found at: <a class="link-external link-https" href="https://github.com/thupchnsky/PoincareLinearClassification" rel="external noopener nofollow">this https URL</a>.

Self-Certifying Classification by Linearized Deep Assignment

Probabilistic Safety Regions Via Finite Families of Scalable Classifiers

Misclassification bounds for PAC-Bayesian sparse deep learning

On Certified Generalization in Structured Prediction

Misclassification Risk and Uncertainty Quantification in Deep Classifiers

De-randomized PAC-Bayes Margin Bounds: Applications to Non-convex and Non-smooth Predictors

Adaptive Hierarchical Certification for Segmentation using Randomized Smoothing

Learning of deep convolutional network image classifiers via stochastic gradient descent and over-parametrization

Structure-Aware Robustness Certificates for Graph Classification

Stochastic Feature Mapping for PAC-Bayes Classification

Robust Certification for Laplace Learning on Geometric Graphs.

Hyperspherical Classification with Dynamic Label-to-Prototype Assignment

Delegated Classification

Provably Accurate and Scalable Linear Classifiers in Hyperbolic Spaces

Risk-aware Classification via Uncertainty Quantification

Regularized Training and Tight Certification for Randomized Smoothed Classifier with Provable Robustness

FullCert: Deterministic End-to-End Certification for Training and Inference of Neural Networks

Data-Driven Performance Guarantees for Classical and Learned Optimizers

Active Learning with Neural Networks: Insights from Nonparametric Statistics

Towards a Framework for Deep Learning Certification in Safety-Critical Applications Using Inherently Safe Design and Run-Time Error Detection

Classification under Nuisance Parameters and Generalized Label Shift in Likelihood-Free Inference