Abstract:Many high-dimensional practical data sets have hierarchical structures induced by graphs or time series. Such data sets are hard to process in Euclidean spaces and one often seeks low-dimensional embeddings in other space forms to perform the required learning tasks. For hierarchical data, the space of choice is a hyperbolic space because it guarantees low-distortion embeddings for tree-like structures. The geometry of hyperbolic spaces has properties not encountered in Euclidean spaces that pose challenges when trying to rigorously analyze algorithmic solutions. We propose a unified framework for learning scalable and simple hyperbolic linear classifiers with provable performance guarantees. The gist of our approach is to focus on Poincaré ball models and formulate the classification problems using tangent space formalisms. Our results include a new hyperbolic perceptron algorithm as well as an efficient and highly accurate convex optimization setup for hyperbolic support vector machine classifiers. Furthermore, we adapt our approach to accommodate second-order perceptrons, where data is preprocessed based on second-order information (correlation) to accelerate convergence, and strategic perceptrons, where potentially manipulated data arrives in an online manner and decisions are made sequentially. The excellent performance of the Poincaré second-order and strategic perceptrons shows that the proposed framework can be extended to general machine learning problems in hyperbolic spaces. Our experimental results, pertaining to synthetic, single-cell RNA-seq expression measurements, CIFAR10, Fashion-MNIST and mini-ImageNet, establish that all algorithms provably converge and have complexity comparable to those of their Euclidean counterparts. Accompanying codes can be found at: <a class="link-external link-https" href="https://github.com/thupchnsky/PoincareLinearClassification" rel="external noopener nofollow">this https URL</a>.

Beyond one-hot encoding: Lower dimensional target embedding

A-Optimal Projection for Image Representation.

Two-Stage Label Embedding Via Neural Factorization Machine for Multi-Label Classification

Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features

High-dimensional Bayesian optimization using low-dimensional feature spaces

Beyond One-Hot-Encoding: Injecting Semantics to Drive Image Classifiers

Target-Embedding Autoencoders for Supervised Representation Learning

Efficient Representation of Low-Dimensional Manifolds using Deep Networks

Enhanced Expressive Power and Fast Training of Neural Networks by Random Projections

Integrating Convolution and Sparse Coding for Learning Low-Dimensional Discriminative Image Representations

Large-Margin Learning of Compact Binary Image Encodings

Beyond Uniform Scaling: Exploring Depth Heterogeneity in Neural Architectures

Investigating the Benefits of Projection Head for Representation Learning

End-to-End Feature-Aware Label Space Encoding for Multilabel Classification with Many Classes.

Joint Learning of Discriminative Low-dimensional Image Representations Based on Dictionary Learning and Two-layer Orthogonal Projections

Improving deep representation learning via auxiliary learnable target coding

Deep Learning Multidimensional Projections

Transductive Multi-View Embedding For Zero-Shot Recognition And Annotation

OneNet: A Channel-Wise 1D Convolutional U-Net

A novel compact design of convolutional layers with spatial transformation towards lower-rank representation for image classification

Provably Accurate and Scalable Linear Classifiers in Hyperbolic Spaces