Abstract:In natural language learning, pre-trained language models (PLM) can acquire rich knowledge and concepts from rich corpora, making it possible to use PLM-based models for knowledge graph completion (KGC) tasks. However, in previous research, when applying pre-trained models to knowledge graph completion tasks, two main challenges persist: (1) Existing knowledge graph completion models are typically evaluated based on the closed-world assumption(CWA), thus lacking evaluation methods suitable for the open-world assumption(OWA), which constitutes a significant challenge in the current field of knowledge graph completion. (2) Extracting useful information, reducing noise, and providing clear interpretability for extracting effective information from the extensive prior knowledge embedded in pre-trained language models is also a crucial issue. Although the loss function can reduce noise to a certain extent, from the perspective of information theory, only relying on the loss function has a limited effect on noise reduction, and the model needs more professional tools to reduce noise and reduce the impact of irrelevant information on model performance. To address the aforementioned challenges, we propose a dual confrontation learning model based on the variational information bottleneck method. This model restricts information flow and feature selection from the perspective of information theory to reduce noise and enhance model performance while providing clear interpretability for this process. Based on extensive experiments and comprehensive evaluations conducted under both closed-world and open-world assumptions, this model successfully extracts valuable knowledge from pre-trained language models to accomplish KGC tasks. Simultaneously, it minimizes noise, removes non-robust features, enhances model reliability, and optimizes model performance. More importantly, we offer a strong interpretability for the process in which our model constrains information flow to reduce noise.

Knowledge-Infused Pre-trained Models for KG Completion

KG-BERT: BERT for Knowledge Graph Completion

Learning to Complete Knowledge Graphs with Deep Sequential Models

Step out of KG: Knowledge Graph Completion via Knowledgeable Retrieval and Reading Comprehension

Exploring Large Language Models for Knowledge Graph Completion

Structure Pre-training and Prompt Tuning for Knowledge Graph Transfer

KERMIT: Knowledge Graph Completion of Enhanced Relation Modeling with Inverse Transformation

Do Pre-trained Models Benefit Knowledge Graph Completion? A Reliable Evaluation and a Reasonable Approach.

Graph Structure Enhanced Pre-Training Language Model for Knowledge Graph Completion

Simple knowledge graph completion model based on PU learning and prompt learning

Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language Models

MEGA: Meta-Graph Augmented Pre-Training Model for Knowledge Graph Completion

A Pre-training Framework for Knowledge Graph Completion

From Discrimination to Generation: Knowledge Graph Completion with Generative Transformer

Fusing Structural Information with Knowledge Enhanced Text Representation for Knowledge Graph Completion

Pre-training Transformers for Knowledge Graph Completion

Does Pre-trained Language Model Actually Infer Unseen Links in Knowledge Graph Completion?

Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models

Knowledge Graph Completing with Dual Confrontation Learning Model Based on Variational Information Bottleneck Method

KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion