Abstract:Human-curated knowledge graphs provide critical supportive information to various natural language processing tasks, but these graphs are usually incomplete, urging auto-completion of them (a.k.a. knowledge graph completion). Prevalent graph embedding approaches, e.g., TransE, learn structured knowledge via representing graph elements (i.e., entities/relations) into dense embeddings and capturing their triple-level relationship with spatial distance. However, they are hardly generalizable to the elements never visited in training and are intrinsically vulnerable to graph incompleteness. In contrast, textual encoding approaches, e.g., KG-BERT, resort to graph triple’s text and triple-level contextualized representations. They are generalizable enough and robust to the incompleteness, especially when coupled with pre-trained encoders. But two major drawbacks limit the performance: (1) high overheads due to the costly scoring of all possible triples in inference, and (2) a lack of structured knowledge in the textual encoder. In this paper, we follow the textual encoding paradigm and aim to alleviate its drawbacks by augmenting it with graph embedding techniques – a complementary hybrid of both paradigms. Specifically, we partition each triple into two asymmetric parts as in translation-based graph embedding approach, and encode both parts into contextualized representations by a Siamese-style textual encoder. Built upon the representations, our model employs both deterministic classifier and spatial measurement for representation and structure learning respectively. It thus reduces the overheads by reusing graph elements’ embeddings to avoid combinatorial explosion, and enhances structured knowledge by exploring the spatial characteristics. Moreover, we develop a self-adaptive ensemble scheme to further improve the performance by incorporating triple scores from an existing graph embedding model. In experiments, we achieve state-of-the-art performance on three benchmarks and a zero-shot dataset for link prediction, with highlights of inference costs reduced by 1-2 orders of magnitude compared to a sophisticated textual encoding method.

Graph Structure Enhanced Pre-Training Language Model for Knowledge Graph Completion

Mixed Geometry Message and Trainable Convolutional Attention Network for Knowledge Graph Completion

MEGA: Meta-Graph Augmented Pre-Training Model for Knowledge Graph Completion

Subgraph-Aware Training of Language Models for Knowledge Graph Completion Using Structure-Aware Contrastive Learning

Fusing Structural Information with Knowledge Enhanced Text Representation for Knowledge Graph Completion

Structure Pre-training and Prompt Tuning for Knowledge Graph Transfer

A Pre-training Framework for Knowledge Graph Completion

KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion

Hierarchical Perceptual Graph Attention Network for Knowledge Graph Completion

Do Pre-trained Models Benefit Knowledge Graph Completion? A Reliable Evaluation and a Reasonable Approach.

Knowledge graph extension with a pre-trained language model via unified learning method

Knowledge Graph Completing with Dual Confrontation Learning Model Based on Variational Information Bottleneck Method

Improving Knowledge Graph Completion with Structure-Aware Supervised Contrastive Learning

Knowledge Graph Completion Method of Combining Structural Information with Semantic Information

Large Models and Multimodal: A Survey of Cutting-Edge Approaches to Knowledge Graph Completion

Simple knowledge graph completion model based on PU learning and prompt learning

Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion

Making Large Language Models Perform Better in Knowledge Graph Completion

Knowledge-Infused Pre-trained Models for KG Completion