Abstract:Semantic networks, such as the knowledge graph, can represent the knowledge leveraging the graph structure. Although the knowledge graph shows promising values in natural language processing, it suffers from incompleteness. This paper focuses on knowledge graph completion by predicting linkage between entities, which is a fundamental yet critical task. Semantic matching is a potential solution as it can deal with unseen entities, which the translational distance based methods struggle with. However, to achieve competitive performance as translational distance based methods, semantic matching based methods require large-scale datasets for the training purpose, which are typically unavailable in practical settings. Therefore, we employ the language model and introduce a novel knowledge graph architecture named LP-BERT, which contains two main stages: multi-task pre-training and knowledge graph fine-tuning. In the pre-training phase, three tasks are taken to drive the model to learn the relationship from triples by predicting either entities or relations. While in the fine-tuning phase, inspired by contrastive learning, we design a triple-style negative sampling in a batch, which greatly increases the proportion of negative sampling while keeping the training time almost unchanged. Furthermore, we propose a new data augmentation method utilizing the inverse relationship of triples to improve the performance and robustness of the model. To demonstrate the effectiveness of our method, we conduct extensive experiments on three widely-used datasets, WN18RR, FB15k-237, and UMLS. The experimental results demonstrate the superiority of our methods, and our approach achieves state-of-the-art results on WN18RR and FB15k-237 datasets. Significantly, Hits@10 indicator is improved by 5% from previous state-of-the-art result on the WN18RR dataset while reaching 100% on the UMLS dataset.

A Novel Multi-Task Learning Framework for Semi-Supervised Semantic Parsing

A Syntax-aware Multi-task Learning Framework for Chinese Semantic Role Labeling

Unsupervised Dual Paraphrasing for Two-stage Semantic Parsing

Dual Learning for Semi-Supervised Natural Language Understanding

Semantic Human Parsing via Scalable Semantic Transfer over Multiple Label Domains

Enhancing Unsupervised Semantic Parsing with Distributed Contextual Representations

Multi-Task Learning for Conversational Question Answering over a Large-Scale Knowledge Base

Unified Semantic Parsing with Weak Supervision

Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing

Merging Weak and Active Supervision for Semantic Parsing

A Span-based Target-aware Relation Model for Frame-semantic Parsing.

Effect of pH on High‐Temperature Production of Bacterial Penicillin Acylase in Escherichiacoli

A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks

Semantic Role Labeling for Learner Chinese: the Importance of Syntactic Parsing and L2-L1 Parallel Data

A Representation Learning Framework For Multi-Source Transfer Parsing

Semi-supervised Multi-task Learning for Semantics and Depth

Semantic Parsing by Large Language Models for Intricate Updating Strategies of Zero-Shot Dialogue State Tracking

Leveraging Multi-Level Semantic Understanding in a Unified NER Model

Multi-task Pre-training Language Model for Semantic Network Completion

Leverage External Knowledge and Self-attention for Chinese Semantic Dependency Graph Parsing