Abstract:Hyperdimensional Computing (HDC) is an emerging computational paradigm for representing compositional information as high-dimensional vectors, and has a promising potential in applications ranging from machine learning to neuromorphic computing. One of the long-standing challenges in HDC is factoring a compositional representation to its constituent factors, also known as the recovery problem. In this paper we take a novel approach to solve the recovery problem, and propose the use of random linear codes. These codes are subspaces over the Boolean field, and are a well-studied topic in information theory with various applications in digital communication. We begin by showing that hyperdimensional encoding using random linear codes retains favorable properties of the prevalent (ordinary) random codes, and hence HD representations using the two methods have comparable information storage capabilities. We proceed to show that random linear codes offer a rich subcode structure that can be used to form key-value stores, which encapsulate most use cases of HDC. Most importantly, we show that under the framework we develop, random linear codes admit simple recovery algorithms to factor (either bundled or bound) compositional representations. The former relies on constructing certain linear equation systems over the Boolean field, the solution to which reduces the search space dramatically and strictly outperforms exhaustive search in many cases. The latter employs the subspace structure of these codes to achieve provably correct factorization. Both methods are strictly faster than the state-of-the-art resonator networks, often by an order of magnitude. We implemented our techniques in Python using a benchmark software library, and demonstrated promising experimental results.

H3DFact: Heterogeneous 3D Integrated CIM for Factorization with Holographic Perceptual Representations

In-memory factorization of holographic perceptual representations

Cognition Math Based on Factor Space

Robust In-Memory Computing with Hyperdimensional Stochastic Representation

Hyperdimensional Quantum Factorization

Depth-assisted calibration on learning-based factorization for a compressive light field display

Generalized Holographic Reduced Representations

Discriminative Concept Factorization for Data Representation

Facit: Factorizing Tensors Into Interpretable And Scrutinizable Patterns

Factor Fields: A Unified Framework for Neural Fields and Beyond

Deep Tensor Factorization for Hyperspectral Image Classification.

Hyperdimensional computing with holographic and adaptive encoder

Deep Matrix Tri-Factorization: Mining Vertex-Wise Interactions In Multi-Space Attributed Graphs

Hyper-graph Regularized Discriminative Concept Factorization for Data Representation

Linear Codes for Hyperdimensional Computing

Towards an Efficient Multi-Way Factorization of Multi-Dimensional Big Data Across A Gpu Cluster

Heterogeneous Matrix Factorization: When Features Differ by Datasets

Divide-Conquer-and-Merge: Memory- and Time-Efficient Holographic Displays

H-PARAFAC: Hierarchical Parallel Factor Analysis of Multidimensional Big Data

In-memory hyperdimensional computing

Factorized Diffusion: Perceptual Illusions by Noise Decomposition