Abstract:Modelling information from complex systems such as humans social interaction or words co-occurrences in our languages can help to understand how these systems are organized and function. Such systems can be modelled by networks, and network theory provides a useful set of methods to analyze them. Among these methods, graph embedding is a powerful tool to summarize the interactions and topology of a network in a vectorized feature space. When used in input of machine learning algorithms, embedding vectors help with common graph problems such as link prediction, graph matching, etc In Natural Language Processing (NLP), such a vectorization process is also employed. Word embedding has the goal of representing the sense of words, extracting it from large text corpora. Despite differences in the structure of information in input of embedding algorithms, many graph embedding approaches are adapted and inspired from methods in NLP. Limits of these methods are observed in both domains. Most of these methods require long and resource greedy training. Another downside to most methods is that they are black-box , from which understanding how the information is structured is rather complex. Interpretability of a model allows understanding how the vector space is structured without the need for external information, and thus can be audited more easily. With both these limitations in mind, we propose a novel framework to efficiently embed network vertices in an interpretable vector space. Our Lower Dimension Bipartite Framework (LDBGF) leverages the bipartite projection of a network using cliques to reduce dimensionality. Along with LDBGF, we introduce two implementations of this framework that rely on communities instead of cliques: SINr-NR and SINr-MF. We show that SINr-MF can perform well on classical graphs and SINr-NR can produce high-quality graph and word embeddings that are interpretable and stable across runs.

Discrete Word Embedding for Logical Natural Language Understanding

Discrete Embedding for Attributed Graphs

Do Multi-Sense Embeddings Improve Natural Language Understanding?

Tsetlin Machine Embedding: Representing Words Using Logical Expressions

Learning Word Sense Embeddings from Word Sense Definitions

Low-dimensional Semantic Space: from Text to Word Embedding

An Exploration Of Semantic Relations In Neural Word Embeddings Using Extrinsic Knowledge

Disentangling continuous and discrete linguistic signals in transformer-based sentence embeddings

Word Equations: Inherently Interpretable Sparse Word Embeddingsthrough Sparse Coding

Towards Invertible Semantic-Preserving Embeddings of Logical Formulae

Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operations

stl2vec: Semantic and Interpretable Vector Representation of Temporal Logic

Language to Logical Form with Neural Attention

From communities to interpretable network and word embedding: an unified approach

Quantum word embedding for machine learning

Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundary

Learning Semantic Representations for Novel Words: Leveraging Both Form and Context

Contextualized word senses: from attention to compositionality

Learning Effective Word Embedding Using Morphological Word Similarity

Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks

Emergent Discrete Communication in Semantic Spaces