Abstract:Knowledge graphs, which consist of entities and their relations, have become a popular way to store structured knowledge. Knowledge graph embedding (KGE), which derives a representation for each entity and relation, has been widely used to capture the semantics of the information in the knowledge graphs, and has demonstrated great success in many downstream applications, such as the extraction of similar entities in response to a query entity. However, existing KGE methods cannot work well on emerging knowledge graphs that are large-scale due to the constraints in storage and inference efficiency. In this paper, we propose a lightweight KGE model, LightKG, which significantly reduces storage as well as running time needed for inference. Instead of storing a continuous vector for every entity, LightKG only needs to store a few codebooks, each of which contains some codewords that correspond to the representatives among the embeddings, and the indices that correspond to the codeword selections for entities. Hence LightKG can achieve highly efficient storage. The efficiency of the downstream querying process can be significantly boosted too with the proposed LightKG model as the relevance score between the query and an entity can be efficiently calculated via a quick look-up in a table that contains the scores between the query and codewords. The storage and inference efficiency of LightKG is achieved by its novel design. LightKG is an end-to-end framework that automatically infers codebooks and codewords and generates an approximated embedding for each entity. A residual module is included in LightKG to induce the diversity among codebooks, and a continuous function is adopted to approximate codeword selection, which is non-differential. In addition, to further improve the performance of KGE, we propose a novel dynamic negative sampling method based on quantization, which can be applied to the proposed LightKG or other KGE methods. We conduct extensive experiments on five public datasets. The experiments show that LightKG is search and memory efficient with high approximate search accuracy. Also, the dynamic negative sampling can dramatically improve model performance with over 19% improvement on average.

PIE: a Parameter and Inference Efficient Solution for Large Scale Knowledge Graph Embedding Reasoning

From Wide to Deep: Dimension Lifting Network for Parameter-efficient Knowledge Graph Embedding

Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning

Knowledge Graph Embedding Via Reasoning over Entities, Relations, and Text

Heuristic-Driven, Type-Specific Embedding in Parallel Spaces for Enhancing Knowledge Graph Reasoning.

Entity-Agnostic Representation Learning for Parameter-Efficient Knowledge Graph Embedding

RIECN: Learning Relation-Based Interactive Embedding Convolutional Network for Knowledge Graph.

Reasoning Through Memorization: Nearest Neighbor Knowledge Graph Embeddings

Efficiently Embedding Dynamic Knowledge Graphs

Sharing Parameter by Conjugation for Knowledge Graph Embeddings in Complex Space

A Lightweight Knowledge Graph Embedding Framework for Efficient Inference and Storage

Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis

CAKE: A Context-Aware Knowledge Embedding Model of Knowledge Graph

Ontological Concept Structure Aware Knowledge Transfer for Inductive Knowledge Graph Embedding.

SEPAKE: a structure-enhanced and position-aware knowledge embedding framework for knowledge graph completion

ProjE: Embedding Projection for Knowledge Graph Completion

A Physical Embedding Model for Knowledge Graphs

Relation Path Embedding in Knowledge Graphs

Multiple Run Ensemble Learning with Low-Dimensional Knowledge Graph Embeddings

Inductively Representing Out-of-Knowledge-Graph Entities by Optimal Estimation under Translational Assumptions

Inference over Unseen Entities, Relations and Literals on Knowledge Graphs