Abstract:High-dimensional distributed semantic spaces have proven useful and effective for aggregating and processing visual, auditory, and lexical information for many tasks related to human-generated data. Human language makes use of a large and varying number of features, lexical and constructional items as well as contextual and discourse-specific data of various types, which all interact to represent various aspects of communicative information. Some of these features are mostly local and useful for the organisation of e.g. argument structure of a predication; others are persistent over the course of a discourse and necessary for achieving a reasonable level of understanding of the content. This paper describes a model for high-dimensional representation for utterance and text level data including features such as constructions or contextual data, based on a mathematically principled and behaviourally plausible approach to representing linguistic information. The implementation of the representation is a straightforward extension of Random Indexing models previously used for lexical linguistic items. The paper shows how the implemented model is able to represent a broad range of linguistic features in a common integral framework of fixed dimensionality, which is computationally habitable, and which is suitable as a bridge between symbolic representations such as dependency analysis and continuous representations used e.g. in classifiers or further machine-learning approaches. This is achieved with operations on vectors that constitute a powerful computational algebra, accompanied with an associative memory for the vectors. The paper provides a technical overview of the framework and a worked through implemented example of how it can be applied to various types of linguistic features.

Towards Using Diachronic Distributed Word Representations as Models of Lexical Development

Diachronic Deviation Features In Continuous Space Word Representations

Using Known Words to Learn More Words: A Distributional Analysis of Child Vocabulary Development

Topic Modeling Using Distributed Word Embeddings

Hierarchical Dirichlet process for tracking complex topical structure evolution and its application to autism research literature

Bilingual Lexicon Extraction with Temporal Distributed Word Representation from Comparable Corpora.

High-dimensional distributed semantic spaces for utterances

Incorporating Linguistic Knowledge for Learning Distributed Word Representations.

Exploring Patterns of Stability and Change in Caregivers' Word Usage Across Early Childhood

Diachronic Synonymy and Polysemy: Exploring Dynamic Relation Between Forms and Meanings of Words Based on Word Embeddings

Learning Topic-Sensitive Word Representations

Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Change

Distribution of words across the first years of life: A longitudinal analysis of everyday language input to three English‐learning infants

Age-Dependent Analysis and Stochastic Generation of Child-Directed Speech

Interpretable Syntactic Representations Enable Hierarchical Word Vectors

A Distributed Representation-Based Framework for Cross-Lingual Transfer Parsing.

A Neural Network Model of Lexical Competition during Infant Spoken Word Recognition

Analyzing language development from a network approach

DRWS: A Model for Learning Distributed Representations for Words and Sentences.

Learning Distributed Representations Of Uyghur Words And Morphemes

A Unified Framework for Jointly Learning Distributed Representations of Word and Attributes.