Knowledge Sheaves: A Sheaf-Theoretic Framework for Knowledge Graph Embedding

Thomas Gebhart,Jakob Hansen,Paul Schrater

DOI: https://doi.org/10.48550/arXiv.2110.03789

2023-03-18

Abstract:Knowledge graph embedding involves learning representations of entities -- the vertices of the graph -- and relations -- the edges of the graph -- such that the resulting representations encode the known factual information represented by the knowledge graph and can be used in the inference of new relations. We show that knowledge graph embedding is naturally expressed in the topological and categorical language of \textit{cellular sheaves}: a knowledge graph embedding can be described as an approximate global section of an appropriate \textit{knowledge sheaf} over the graph, with consistency constraints induced by the knowledge graph's schema. This approach provides a generalized framework for reasoning about knowledge graph embedding models and allows for the expression of a wide range of prior constraints on embeddings. Further, the resulting embeddings can be easily adapted for reasoning over composite relations without special training. We implement these ideas to highlight the benefits of the extensions inspired by this new perspective.

Machine Learning,Algebraic Topology

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

The paper attempts to address several key issues in Knowledge Graph Embedding (KGE):

1. **Unified Framework**: Existing knowledge graph embedding methods are numerous, each with its unique motivations and theoretical foundations. The paper proposes a unified framework based on cellular sheaves, which can incorporate various existing KGE methods, thereby providing a unified language to understand and extend these methods.

2. **Global Consistency**: One of the goals of knowledge graph embedding is to maintain both local and global consistency of the graph in the embedding space. The paper provides a formal language through the theory of cellular sheaves to express and optimize these consistency constraints.

3. **Prior Constraints**: Many existing KGE methods lack effective modeling of prior knowledge about entities and relationships. The proposed method allows the introduction of various prior constraints during the embedding process, such as symmetry, asymmetry, one-to-one, and one-to-many relationship types, thereby better capturing the semantics of the underlying knowledge domain.

4. **Multi-hop Reasoning**: An important application of knowledge graph embedding is multi-hop reasoning, which involves inferring complex relationships by combining basic ones. The paper utilizes the sheaf Laplacian to provide a natural method for handling multi-hop queries.

5. **Generalization Ability**: To enable knowledge graph embeddings to be applied to unseen data in downstream tasks, the paper proposes a method for learning multiple approximate sections to improve the robustness and generalization ability of the embeddings.

### Abstract Summary

Knowledge graph embedding involves learning representations of entities (vertices of the graph) and relationships (edges of the graph) such that these representations can encode known factual information in the knowledge graph and be used to infer new relationships. The paper demonstrates that knowledge graph embeddings can be naturally expressed using the topology and categorical language of cellular sheaves, describing knowledge graph embeddings as approximate global sections of appropriate knowledge sheaves on the graph, induced by consistency constraints from the knowledge graph's schema. This framework provides a general approach for reasoning about knowledge graph embedding models and allows for the expression of a wide range of embedding prior constraints. Additionally, the generated embeddings can easily adapt to reasoning about composite relationships without special training. The paper implements these ideas, highlighting the benefits brought by this new perspective.