Abstract:Semantic networks, such as the knowledge graph, can represent the knowledge leveraging the graph structure. Although the knowledge graph shows promising values in natural language processing, it suffers from incompleteness. This paper focuses on knowledge graph completion by predicting linkage between entities, which is a fundamental yet critical task. Semantic matching is a potential solution as it can deal with unseen entities, which the translational distance based methods struggle with. However, to achieve competitive performance as translational distance based methods, semantic matching based methods require large-scale datasets for the training purpose, which are typically unavailable in practical settings. Therefore, we employ the language model and introduce a novel knowledge graph architecture named LP-BERT, which contains two main stages: multi-task pre-training and knowledge graph fine-tuning. In the pre-training phase, three tasks are taken to drive the model to learn the relationship from triples by predicting either entities or relations. While in the fine-tuning phase, inspired by contrastive learning, we design a triple-style negative sampling in a batch, which greatly increases the proportion of negative sampling while keeping the training time almost unchanged. Furthermore, we propose a new data augmentation method utilizing the inverse relationship of triples to improve the performance and robustness of the model. To demonstrate the effectiveness of our method, we conduct extensive experiments on three widely-used datasets, WN18RR, FB15k-237, and UMLS. The experimental results demonstrate the superiority of our methods, and our approach achieves state-of-the-art results on WN18RR and FB15k-237 datasets. Significantly, Hits@10 indicator is improved by 5% from previous state-of-the-art result on the WN18RR dataset while reaching 100% on the UMLS dataset.

Neural Multi-Task Learning for Citation Function and Provenance

Convolutional Neural Networks Based Multi-task Deep Learning for Movie Review Classification

Citation Metadata Extraction Via Deep Neural Network-based Segment Sequence Labeling

Neural Multi-Task Learning for Stance Prediction

Meta Multi-Task Learning for Sequence Modeling.

AutoCite: Multi-Modal Representation Fusion for Contextual Citation Generation

Adversarial Multi-task Learning for Text Classification.

A Multi-task Learning Approach for Image Captioning.

Multi-Task CNN Model for Attribute Prediction

Recurrent Neural Network for Text Classification with Multi-Task Learning

Low-Rank Deep Convolutional Neural Network for Multi-Task Learning

A Self-Attentional Neural Architecture for Code Completion with Multi-Task Learning.

A Multi-Task Learning Neural Network for Emotion-Cause Pair Extraction.

Resolving Citation Links With Neural Networks

Deep Multi-Task Learning with Shared Memory for Text Classification

Multi-Task Learning Improves Performance In Deep Argument Mining Models

Learn from Your Neighbor: Learning Multi-modal Mappings from Sparse Annotations

Meta-Learning Multi-task Communication.

Multi-Task Learning Model Based on Multi-Scale CNN and LSTM for Sentiment Classification

Multi-task Learning Based Neural Bridging Reference Resolution

Multi-task Pre-training Language Model for Semantic Network Completion