Abstract:A vast amount of textual and structural information is required for knowledge graph construction and its downstream tasks. However, most of the current knowledge graphs are incomplete due to the difficulty of knowledge acquisition and integration. Knowledge Graph Completion (KGC) is used to predict missing connections. In previous studies, textual information and graph structural information are utilized independently, without an effective method for fusing these two types of information. In this paper, we propose a graph structure enhanced pre-training language model for knowledge graph completion. Firstly, we design a graph sampling algorithm and a Graph2Seq module for constructing sub-graphs and their corresponding contexts to support large-scale knowledge graph learning and parallel training. It is also the basis for fusing textual data and graph structure. Next, two pre-training tasks based on masked modeling are designed for capturing accurate entity-level and relation-level information. Furthermore, this paper proposes a novel asymmetric Encoder-Decoder architecture to restore masked components, where the encoder is a Pre-trained Language Model (PLM) and the decoder is a multi-relational Graph Neural Network (GNN). The purpose of the architecture is to integrate textual information effectively with graph structural information. Finally, the model is fine-tuned for KGC tasks on two widely used public datasets. The experiments show that the model achieves excellent performance and outperforms baselines in most metrics, which demonstrate the effectiveness of our approach by fusing the structure and semantic information to knowledge graph.

Generative adversarial meta-learning knowledge graph completion for large-scale complex knowledge graphs

Mixed Geometry Message and Trainable Convolutional Attention Network for Knowledge Graph Completion

Exploring Large Language Models for Knowledge Graph Completion

Learning Structured Embeddings of Knowledge Graphs with Generative Adversarial Framework

Generative Transformer with Knowledge-Guided Decoding for Academic Knowledge Graph Completion

Multilingual Knowledge Graph Completion with Self-Supervised Adaptive Graph Alignment

Large Models and Multimodal: A Survey of Cutting-Edge Approaches to Knowledge Graph Completion

CogMG: Collaborative Augmentation Between Large Language Model and Knowledge Graph

Meta-Learning Based Dynamic Adaptive Relation Learning for Few-Shot Knowledge Graph Completion

Graph Structure Enhanced Pre-Training Language Model for Knowledge Graph Completion

Hierarchical Perceptual Graph Attention Network for Knowledge Graph Completion

Simple and effective meta relational learning for few-shot knowledge graph completion

From Discrimination to Generation: Knowledge Graph Completion with Generative Transformer

GRL: Knowledge graph completion with GAN-based reinforcement learning

LLM-based Multi-Level Knowledge Generation for Few-shot Knowledge Graph Completion

Learning Structured Embeddings of Knowledge Graphs with Adversarial Learning Framework

A knowledge graph completion model based on contrastive learning and relation enhancement method

GLaM: Fine-Tuning Large Language Models for Domain Knowledge Graph Alignment via Neighborhood Partitioning and Generative Subgraph Encoding

Better Together: Enhancing Generative Knowledge Graph Completion with Language Models and Neighborhood Information

In-Context Learning with Topological Information for Knowledge Graph Completion

Dual Channel Knowledge Graph Embedding with Ontology Guided Data Augmentation.