Abstract:We design a new technique for the distributional semantic modeling with a neural network-based approach to learn distributed term representations (or term embeddings) - term vector space models as a result, inspired by the recent ontology-related approach (using different types of contextual knowledge such as syntactic knowledge, terminological knowledge, semantic knowledge, etc.) to the identification of terms (term extraction) and relations between them (relation extraction) called semantic pre-processing technology - SPT. Our method relies on automatic term extraction from the natural language texts and subsequent formation of the problem-oriented or application-oriented (also deeply annotated) text corpora where the fundamental entity is the term (includes non-compositional and compositional terms). This gives us an opportunity to changeover from distributed word representations (or word embeddings) to distributed term representations (or term embeddings). This transition will allow to generate more accurate semantic maps of different subject domains (also, of relations between input terms - it is useful to explore clusters and oppositions, or to test your hypotheses about them). The semantic map can be represented as a graph using Vec2graph - a Python library for visualizing word embeddings (term embeddings in our case) as dynamic and interactive graphs. The Vec2graph library coupled with term embeddings will not only improve accuracy in solving standard NLP tasks, but also update the conventional concept of automated ontology development. The main practical result of our work is the development kit (set of toolkits represented as web service APIs and web application), which provides all necessary routines for the basic linguistic pre-processing and the semantic pre-processing of the natural language texts in Ukrainian for future training of term vector space models.

Collaborative Training of Tensors for Compositional Distributional Semantics

Phrase Type Sensitive Tensor Indexing Model for Semantic Composition

Collaborative Topic Modeling for Text Tensors

Distributional semantic modeling: a revised technique to train term/word vector space models applying the ontology-related approach

Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning

Language Modeling Using Tensor Trains

Joint Semantic Synthesis and Morphological Analysis of the Derived Word

Distributive Pre-Training of Generative Modeling Using Matrix-Product States

Distributional Models and Deep Learning Embeddings: Combining the Best of Both Worlds

Tensor Decompositions Meet Control Theory: Learning General Mixtures of Linear Dynamical Systems

Compositional Operators in Distributional Semantics

Embedding Syntax and Semantics of Prepositions via Tensor Decomposition

Tensor-based transductive learning for multimodality video semantic concept detection

Latent Space Disentanglement in Diffusion Transformers Enables Zero-shot Fine-grained Semantic Editing

Compositionality as Lexical Symmetry

Learning Mixed Membership Community Models in Social Tagging Networks through Tensor Methods

Modelling Lexical Ambiguity with Density Matrices

Supervised Learning for Non-Sequential Data: A Canonical Polyadic Decomposition Approach

Event Representations With Tensor-Based Compositions

High-dimensional distributed semantic spaces for utterances

Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training