Abstract:Semantic networks, such as the knowledge graph, can represent the knowledge leveraging the graph structure. Although the knowledge graph shows promising values in natural language processing, it suffers from incompleteness. This paper focuses on knowledge graph completion by predicting linkage between entities, which is a fundamental yet critical task. Semantic matching is a potential solution as it can deal with unseen entities, which the translational distance based methods struggle with. However, to achieve competitive performance as translational distance based methods, semantic matching based methods require large-scale datasets for the training purpose, which are typically unavailable in practical settings. Therefore, we employ the language model and introduce a novel knowledge graph architecture named LP-BERT, which contains two main stages: multi-task pre-training and knowledge graph fine-tuning. In the pre-training phase, three tasks are taken to drive the model to learn the relationship from triples by predicting either entities or relations. While in the fine-tuning phase, inspired by contrastive learning, we design a triple-style negative sampling in a batch, which greatly increases the proportion of negative sampling while keeping the training time almost unchanged. Furthermore, we propose a new data augmentation method utilizing the inverse relationship of triples to improve the performance and robustness of the model. To demonstrate the effectiveness of our method, we conduct extensive experiments on three widely-used datasets, WN18RR, FB15k-237, and UMLS. The experimental results demonstrate the superiority of our methods, and our approach achieves state-of-the-art results on WN18RR and FB15k-237 datasets. Significantly, Hits@10 indicator is improved by 5% from previous state-of-the-art result on the WN18RR dataset while reaching 100% on the UMLS dataset.

Improving Relation Extraction by Knowledge Representation Learning

Knowledge Representation Learning with Entity Descriptions, Hierarchical Types, and Textual Relations

Enriching Pre-trained Language Model with Entity Information for Relation Classification

Rescue Implicit and Long-tail Cases: Nearest Neighbor Relation Extraction

KICE: A Knowledge Consolidation and Expansion Framework for Relation Extraction.

Improving Neural Relation Extraction with Implicit Mutual Relations

REKER: Relation Extraction with Knowledge of Entity and Relation.

Knowledge-Aware And Retrieval-Based Models For Distantly Supervised Relation Extraction

End-to-end Named Entity Recognition and Relation Extraction using Pre-trained Language Models

Improving Relation Extraction with Relational Paraphrase Sentences.

Improving Recall of Large Language Models: A Model Collaboration Approach for Relational Triple Extraction

Context and Type Enhanced Representation Learning for Relation Extraction.

Neural Relation Extraction for Knowledge Base Enrichment

Learning Relation Prototype from Unlabeled Texts for Long-Tail Relation Extraction

Relation of the Relations: A New Paradigm of the Relation Extraction Problem

KEHRL: Learning Knowledge-Enhanced Language Representations with Hierarchical Reinforcement Learning

Leveraging Knowledge Graph Embeddings to Enhance Contextual Representations for Relation Extraction

Improving Relation Extraction by Pre-trained Language Representations

Knowledge Representation Learning with Entities, Attributes and Relations.

Multi-task Pre-training Language Model for Semantic Network Completion

Knowledge-embodied Attention for Distantly Supervised Relation Extraction