Abstract:Modelling information from complex systems such as humans social interaction or words co-occurrences in our languages can help to understand how these systems are organized and function. Such systems can be modelled by networks, and network theory provides a useful set of methods to analyze them. Among these methods, graph embedding is a powerful tool to summarize the interactions and topology of a network in a vectorized feature space. When used in input of machine learning algorithms, embedding vectors help with common graph problems such as link prediction, graph matching, etc In Natural Language Processing (NLP), such a vectorization process is also employed. Word embedding has the goal of representing the sense of words, extracting it from large text corpora. Despite differences in the structure of information in input of embedding algorithms, many graph embedding approaches are adapted and inspired from methods in NLP. Limits of these methods are observed in both domains. Most of these methods require long and resource greedy training. Another downside to most methods is that they are black-box , from which understanding how the information is structured is rather complex. Interpretability of a model allows understanding how the vector space is structured without the need for external information, and thus can be audited more easily. With both these limitations in mind, we propose a novel framework to efficiently embed network vertices in an interpretable vector space. Our Lower Dimension Bipartite Framework (LDBGF) leverages the bipartite projection of a network using cliques to reduce dimensionality. Along with LDBGF, we introduce two implementations of this framework that rely on communities instead of cliques: SINr-NR and SINr-MF. We show that SINr-MF can perform well on classical graphs and SINr-NR can produce high-quality graph and word embeddings that are interpretable and stable across runs.

Online Learning of Interpretable Word Embeddings

Word Embedding Revisited: A New Representation Learning and Explicit Matrix Factorization Perspective.

Learning Word Embeddings from Intrinsic and Extrinsic Views

Interpretable Neural Embeddings with Sparse Self-Representation

Sparse word embeddings using l1 regularized online learning

Learning Cross-lingual Word Embeddings Via Matrix Co-factorization.

Interpreting Embedding Spaces by Conceptualization

Word Equations: Inherently Interpretable Sparse Word Embeddingsthrough Sparse Coding

A Deep Non-negative Matrix Factorization Model for Big Data Representation Learning

Compressing and Interpreting Word Embeddings with Latent Space Regularization and Interactive Semantics Probing

From communities to interpretable network and word embedding: an unified approach

Fast Extraction of Word Embedding from Q-contexts

A Generative Word Embedding Model and its Low Rank Positive Semidefinite Solution

Tackling Interpretability in Audio Classification Networks with Non-negative Matrix Factorization

EmbedLLM: Learning Compact Representations of Large Language Models

Learning Context-Specific Word/Character Embeddings.

Hyperpolyglot LLMs: Cross-Lingual Interpretability in Token Embeddings

Jointly Modeling Embedding and Translation to Bridge Video and Language

Improving interpretability of word embeddings by generating definition and usage

Group-Sparse Matrix Factorization for Transfer Learning of Word Embeddings