Abstract:High-dimensional distributed semantic spaces have proven useful and effective for aggregating and processing visual, auditory, and lexical information for many tasks related to human-generated data. Human language makes use of a large and varying number of features, lexical and constructional items as well as contextual and discourse-specific data of various types, which all interact to represent various aspects of communicative information. Some of these features are mostly local and useful for the organisation of e.g. argument structure of a predication; others are persistent over the course of a discourse and necessary for achieving a reasonable level of understanding of the content. This paper describes a model for high-dimensional representation for utterance and text level data including features such as constructions or contextual data, based on a mathematically principled and behaviourally plausible approach to representing linguistic information. The implementation of the representation is a straightforward extension of Random Indexing models previously used for lexical linguistic items. The paper shows how the implemented model is able to represent a broad range of linguistic features in a common integral framework of fixed dimensionality, which is computationally habitable, and which is suitable as a bridge between symbolic representations such as dependency analysis and continuous representations used e.g. in classifiers or further machine-learning approaches. This is achieved with operations on vectors that constitute a powerful computational algebra, accompanied with an associative memory for the vectors. The paper provides a technical overview of the framework and a worked through implemented example of how it can be applied to various types of linguistic features.

Low-dimensional Semantic Space: from Text to Word Embedding

Contextual Text Understanding in Distributional Semantic Space.

Semantic Word Cloud Generation Based on Word Embeddings

A Survey of Word Embeddings Based on Deep Learning

Semantic Visualization for Short Texts with Word Embeddings

High-dimensional distributed semantic spaces for utterances

An Exploration Of Semantic Relations In Neural Word Embeddings Using Extrinsic Knowledge

Word Embedding for Social Sciences: An Interdisciplinary Survey

Investigating Language Universal and Specific Properties in Word Embeddings

Latent Space Cartography: Visual Analysis of Vector Space Embeddings

Analyzing Structures in the Semantic Vector Space: A Framework for Decomposing Word Embeddings

Learning Word Sense Embeddings from Word Sense Definitions

Semantic Structure-Based Word Embedding by Incorporating Concept Convergence and Word Divergence

Compressing and Interpreting Word Embeddings with Latent Space Regularization and Interactive Semantics Probing

The Semantic Mapping of Words and Co-Words in Contexts

Learning Continuous Word Representations from Large-Scale Corpus Through Linear Approach.

Enhancing Semantic Word Representations by Embedding Deeper Word Relationships

Learning Semantic Representations for Novel Words: Leveraging Both Form and Context

Analysing the Semantic Change Based on Word Embedding

A Survey of Word Embedding Algorithms for Textual Data Information Extraction

Semantic Hilbert Space for Text Representation Learning