Abstract:Vector space models for symbolic processing that encode symbols by random vectors have been proposed in cognitive science and connectionist communities under the names Vector Symbolic Architecture (VSA), and, synonymously, Hyperdimensional (HD) computing. In this paper, we generalize VSAs to function spaces by mapping continuous-valued data into a vector space such that the inner product between the representations of any two data points represents a similarity kernel. By analogy to VSA, we call this new function encoding and computing framework Vector Function Architecture (VFA). In VFAs, vectors can represent individual data points as well as elements of a function space (a reproducing kernel Hilbert space). The algebraic vector operations, inherited from VSA, correspond to well-defined operations in function space. Furthermore, we study a previously proposed method for encoding continuous data, fractional power encoding (FPE), which uses exponentiation of a random base vector to produce randomized representations of data points and fulfills the kernel properties for inducing a VFA. We show that the distribution from which elements of the base vector are sampled determines the shape of the FPE kernel, which in turn induces a VFA for computing with band-limited functions. In particular, VFAs provide an algebraic framework for implementing large-scale kernel machines with random features, extending Rahimi and Recht, 2007. Finally, we demonstrate several applications of VFA models to problems in image recognition, density estimation and nonlinear regression. Our analyses and results suggest that VFAs constitute a powerful new framework for representing and manipulating functions in distributed neural systems, with myriad applications in artificial intelligence.

Self-Attention Based Semantic Decomposition in Vector Symbolic Architectures

Residual and Attentional Architectures for Vector-Symbols

LARS-VSA: A Vector Symbolic Architecture For Learning with Abstract Rules

A Walsh Hadamard Derived Linear Vector Symbolic Architecture

Analyzing Structures in the Semantic Vector Space: A Framework for Decomposing Word Embeddings

Cognitive modeling and learning with sparse binary hypervectors

Vector Symbolic Architectures as a Computing Framework for Emerging Hardware

Hyperseed: Unsupervised Learning with Vector Symbolic Architectures

Hyperdimensional Computing with Spiking-Phasor Neurons

RESOLVE: Relational Reasoning with Symbolic and Object-Level Features Using Vector Symbolic Processing

Computing on Functions Using Randomized Vector Representations

Vector Symbolic Architectures for Context-Free Grammars

A neuro-vector-symbolic architecture for solving Raven's progressive matrices

VSA4VQA: Scaling a Vector Symbolic Architecture to Visual Question Answering on Natural Images

Disentangling semantics in language through VAEs and a certain architectural choice

Semantic Vector Spaces for Broadening Consideration of Consequences

Vector-based Representation is the Key: A Study on Disentanglement and Compositional Generalization

GPT-2 Through the Lens of Vector Symbolic Architectures

Discrete Memory Addressing Variational Autoencoder for Visual Concept Learning

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations

Unsupervised Learning of Neurosymbolic Encoders