Abstract:High-dimensional distributed semantic spaces have proven useful and effective for aggregating and processing visual, auditory, and lexical information for many tasks related to human-generated data. Human language makes use of a large and varying number of features, lexical and constructional items as well as contextual and discourse-specific data of various types, which all interact to represent various aspects of communicative information. Some of these features are mostly local and useful for the organisation of e.g. argument structure of a predication; others are persistent over the course of a discourse and necessary for achieving a reasonable level of understanding of the content. This paper describes a model for high-dimensional representation for utterance and text level data including features such as constructions or contextual data, based on a mathematically principled and behaviourally plausible approach to representing linguistic information. The implementation of the representation is a straightforward extension of Random Indexing models previously used for lexical linguistic items. The paper shows how the implemented model is able to represent a broad range of linguistic features in a common integral framework of fixed dimensionality, which is computationally habitable, and which is suitable as a bridge between symbolic representations such as dependency analysis and continuous representations used e.g. in classifiers or further machine-learning approaches. This is achieved with operations on vectors that constitute a powerful computational algebra, accompanied with an associative memory for the vectors. The paper provides a technical overview of the framework and a worked through implemented example of how it can be applied to various types of linguistic features.

On the difficulty of a distributional semantics of spoken language

Unsupervised Language Acquisition

Introducing Semantics into Speech Encoders.

Distributional Semantics: Meaning Through Culture and Interaction

Learning semantic correspondences with less supervision

Investigating large language models for their competence in extracting grammatically sound sentences from transcribed noisy utterances

A Weakly Supervised Learning Approach for Spoken Language Understanding.

A Distributional Semantics Approach to Implicit Language Learning

Bridging the Gap Between Clean Data Training and Real-World Inference for Spoken Language Understanding

Investigating Low-Cost LLM Annotation for~Spoken Dialogue Understanding Datasets

Are Discrete Units Necessary for Spoken Language Modeling?

Learning to Understand Child-directed and Adult-directed Speech

Semi-Supervised Spoken Language Understanding Via Self-Supervised Speech and Language Model Pretraining.

Robust Spoken Language Understanding with Acoustic and Domain Knowledge

Learning Fine-Grained Semantics in Spoken Language Using Visual Grounding

Modeling speech recognition and synthesis simultaneously: Encoding and decoding lexical and sublexical semantic information into speech with no direct access to speech data

Challenges for Distributional Compositional Semantics

Spoken Language Understanding from Unaligned Data Using Discriminative Classification Models

Spatial versus graphical representation of distributional semantic knowledge.

Understanding Semantics from Speech Through Pre-training

High-dimensional distributed semantic spaces for utterances