Abstract:Human-curated knowledge graphs provide critical supportive information to various natural language processing tasks, but these graphs are usually incomplete, urging auto-completion of them (a.k.a. knowledge graph completion). Prevalent graph embedding approaches, e.g., TransE, learn structured knowledge via representing graph elements (i.e., entities/relations) into dense embeddings and capturing their triple-level relationship with spatial distance. However, they are hardly generalizable to the elements never visited in training and are intrinsically vulnerable to graph incompleteness. In contrast, textual encoding approaches, e.g., KG-BERT, resort to graph triple’s text and triple-level contextualized representations. They are generalizable enough and robust to the incompleteness, especially when coupled with pre-trained encoders. But two major drawbacks limit the performance: (1) high overheads due to the costly scoring of all possible triples in inference, and (2) a lack of structured knowledge in the textual encoder. In this paper, we follow the textual encoding paradigm and aim to alleviate its drawbacks by augmenting it with graph embedding techniques – a complementary hybrid of both paradigms. Specifically, we partition each triple into two asymmetric parts as in translation-based graph embedding approach, and encode both parts into contextualized representations by a Siamese-style textual encoder. Built upon the representations, our model employs both deterministic classifier and spatial measurement for representation and structure learning respectively. It thus reduces the overheads by reusing graph elements’ embeddings to avoid combinatorial explosion, and enhances structured knowledge by exploring the spatial characteristics. Moreover, we develop a self-adaptive ensemble scheme to further improve the performance by incorporating triple scores from an existing graph embedding model. In experiments, we achieve state-of-the-art performance on three benchmarks and a zero-shot dataset for link prediction, with highlights of inference costs reduced by 1-2 orders of magnitude compared to a sophisticated textual encoding method.

Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion

Fusing Structural Information with Knowledge Enhanced Text Representation for Knowledge Graph Completion

SEPAKE: a structure-enhanced and position-aware knowledge embedding framework for knowledge graph completion

Knowledge Graph Embedding with Diversity of Structures

Federated Knowledge Graph Completion Via Embedding-Contrastive Learning

Knowledge Graph Completion Method of Combining Structural Information with Semantic Information

Graph Structure Enhanced Pre-Training Language Model for Knowledge Graph Completion

Convolutional Network Embedding of Text-Enhanced Representation for Knowledge Graph Completion

Bridge: A Unified Framework to Knowledge Graph Completion via Language Models and Knowledge Representation

SSKGE: a time-saving knowledge graph embedding framework based on structure enhancement and semantic guidance

Knowledge Graph Completion using Structural and Textual Embeddings

Text-Graph Enhanced Knowledge Graph Representation Learning.

Siamese Pre-Trained Transformer Encoder for Knowledge Base Completion

Semantic-Enhanced Knowledge Graph Completion

Joint Language Semantic and Structure Embedding for Knowledge Graph Completion.

Efficient Relational Context Perception for Knowledge Graph Completion

KGEL: A novel end-to-end embedding learning framework for knowledge graph completion

KERMIT: Knowledge Graph Completion of Enhanced Relation Modeling with Inverse Transformation

Leveraging Lexical Semantic Information for Learning Concept-Based Multiple Embedding Representations for Knowledge Graph Completion.

Simple knowledge graph completion model based on PU learning and prompt learning

Text-Augmented Knowledge Representation Learning Based on Convolutional Network