Abstract:Many high-dimensional practical data sets have hierarchical structures induced by graphs or time series. Such data sets are hard to process in Euclidean spaces and one often seeks low-dimensional embeddings in other space forms to perform the required learning tasks. For hierarchical data, the space of choice is a hyperbolic space because it guarantees low-distortion embeddings for tree-like structures. The geometry of hyperbolic spaces has properties not encountered in Euclidean spaces that pose challenges when trying to rigorously analyze algorithmic solutions. We propose a unified framework for learning scalable and simple hyperbolic linear classifiers with provable performance guarantees. The gist of our approach is to focus on Poincaré ball models and formulate the classification problems using tangent space formalisms. Our results include a new hyperbolic perceptron algorithm as well as an efficient and highly accurate convex optimization setup for hyperbolic support vector machine classifiers. Furthermore, we adapt our approach to accommodate second-order perceptrons, where data is preprocessed based on second-order information (correlation) to accelerate convergence, and strategic perceptrons, where potentially manipulated data arrives in an online manner and decisions are made sequentially. The excellent performance of the Poincaré second-order and strategic perceptrons shows that the proposed framework can be extended to general machine learning problems in hyperbolic spaces. Our experimental results, pertaining to synthetic, single-cell RNA-seq expression measurements, CIFAR10, Fashion-MNIST and mini-ImageNet, establish that all algorithms provably converge and have complexity comparable to those of their Euclidean counterparts. Accompanying codes can be found at: <a class="link-external link-https" href="https://github.com/thupchnsky/PoincareLinearClassification" rel="external noopener nofollow">this https URL</a>.

On the Transformation Mechanisms of Multilayer Perceptrons with Sigmoid Activation Functions for Classifications

Feature Importance Measure of a Multilayer Perceptron Based on the Presingle-Connection Layer

Classification Ability of Single Hidden Layer Feedforward Neural Networks

Convergence and objective functions of noise-injected multilayer perceptrons with hidden multipliers

Multilayer neural networks with extensively many hidden units

Transparent Classification with Multilayer Logical Perceptrons and Random Binarization

Analytical Form Of Fisher Information Matrix Of Bipoloar-Activation-Function-Based Multilayer Perceptrons

Morphological Perceptrons: Geometry and Training Algorithms

A Probabilistic Representation of Deep Learning for Improving The Information Theoretic Interpretability

Heterogeneous Multilayer Generalized Operational Perceptron

Polyhedrons and Perceptrons Are Functionally Equivalent

On the Importance of Normalisation Layers in Deep Learning with Piecewise Linear Activation Units

Local linear perceptrons for classification

Efficient Estimation of Multidimensional Regression Model using Multilayer Perceptrons

Provably Accurate and Scalable Linear Classifiers in Hyperbolic Spaces

Hidden Unit Specialization in Layered Neural Networks: ReLU vs. Sigmoidal Activation

Activation Functions for Generalized Learning Vector Quantization - A Performance Comparison

A novel Mathematical Modeling for Deep Multilayer Perceptron Optimization: Architecture Optimization and Activation Functions Selection

On the capabilities of multilayer perceptrons

A Smooth Optimisation Perspective on Designing and Training Feedforward Multilayer Perceptrons

Generalization ability of a perceptron with non-monotonic transfer function