Abstract:Hyperdimensional Computing (HDC) is an emerging computational paradigm for representing compositional information as high-dimensional vectors, and has a promising potential in applications ranging from machine learning to neuromorphic computing. One of the long-standing challenges in HDC is factoring a compositional representation to its constituent factors, also known as the recovery problem. In this paper we take a novel approach to solve the recovery problem, and propose the use of random linear codes. These codes are subspaces over the Boolean field, and are a well-studied topic in information theory with various applications in digital communication. We begin by showing that hyperdimensional encoding using random linear codes retains favorable properties of the prevalent (ordinary) random codes, and hence HD representations using the two methods have comparable information storage capabilities. We proceed to show that random linear codes offer a rich subcode structure that can be used to form key-value stores, which encapsulate most use cases of HDC. Most importantly, we show that under the framework we develop, random linear codes admit simple recovery algorithms to factor (either bundled or bound) compositional representations. The former relies on constructing certain linear equation systems over the Boolean field, the solution to which reduces the search space dramatically and strictly outperforms exhaustive search in many cases. The latter employs the subspace structure of these codes to achieve provably correct factorization. Both methods are strictly faster than the state-of-the-art resonator networks, often by an order of magnitude. We implemented our techniques in Python using a benchmark software library, and demonstrated promising experimental results.

A Theoretical Perspective on Hyperdimensional Computing

Classification using Hyperdimensional Computing: A Review

Robust In-Memory Computing with Hyperdimensional Stochastic Representation

Hyperdimensional computing: A fast, robust, and interpretable paradigm for biological data

In-memory hyperdimensional computing

Hyperdimensional computing: a fast, robust and interpretable paradigm for biological data

RegHD: Robust and Efficient Regression in Hyper-Dimensional Learning System

Generalized Holographic Reduced Representations

Task-Projected Hyperdimensional Computing for Multi-Task Learning

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations

Laplace-HDC: Understanding the geometry of binary hyperdimensional computing

Efficient Hyperdimensional Computing

Hyperdimensional Computing vs. Neural Networks: Comparing Architecture and Learning Process

SearcHD: A Memory-Centric Hyperdimensional Computing with Stochastic Training

Efficient Hyperdimensional Computing With Spiking Phasors

uHD: Unary Processing for Lightweight and Dynamic Hyperdimensional Computing

Hyperdimensional computing with holographic and adaptive encoder

Computing with Residue Numbers in High-Dimensional Representation

Linear Codes for Hyperdimensional Computing

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

Energy Efficient In-memory Hyperdimensional Encoding for Spatio-temporal Signal Processing