Abstract:Semantic networks, such as the knowledge graph, can represent the knowledge leveraging the graph structure. Although the knowledge graph shows promising values in natural language processing, it suffers from incompleteness. This paper focuses on knowledge graph completion by predicting linkage between entities, which is a fundamental yet critical task. Semantic matching is a potential solution as it can deal with unseen entities, which the translational distance based methods struggle with. However, to achieve competitive performance as translational distance based methods, semantic matching based methods require large-scale datasets for the training purpose, which are typically unavailable in practical settings. Therefore, we employ the language model and introduce a novel knowledge graph architecture named LP-BERT, which contains two main stages: multi-task pre-training and knowledge graph fine-tuning. In the pre-training phase, three tasks are taken to drive the model to learn the relationship from triples by predicting either entities or relations. While in the fine-tuning phase, inspired by contrastive learning, we design a triple-style negative sampling in a batch, which greatly increases the proportion of negative sampling while keeping the training time almost unchanged. Furthermore, we propose a new data augmentation method utilizing the inverse relationship of triples to improve the performance and robustness of the model. To demonstrate the effectiveness of our method, we conduct extensive experiments on three widely-used datasets, WN18RR, FB15k-237, and UMLS. The experimental results demonstrate the superiority of our methods, and our approach achieves state-of-the-art results on WN18RR and FB15k-237 datasets. Significantly, Hits@10 indicator is improved by 5% from previous state-of-the-art result on the WN18RR dataset while reaching 100% on the UMLS dataset.

Applying a Multitask Feature Sparsity Method for the Classification of Semantic Relations Between Nominals

Model Semantic Relations with Extended Attributes

Hierarchical and Bidirectional Joint Multi-Task Classifiers for Natural Language Understanding

Improving Short Text Classification Through Better Feature Space Selection

Semantic-specific multimodal relation learning for sentiment analysis

Bridging the Emotional Semantic Gap via Multimodal Relevance Estimation

Incorporating multivariate semantic association graphs into multimodal networks for information extraction from documents

Semantic Relation Classification Via Convolutional Neural Networks with Simple Negative Sampling

Multi-modal Learning with Missing Modality via Shared-Specific Feature Modelling

RE-Matching: A Fine-Grained Semantic Matching Method for Zero-Shot Relation Extraction.

Relation Classification Via CNN, Segmented Max-pooling, and SDP-BLSTM.

Semisupervised Feature Analysis by Mining Correlations among Multiple Tasks.

Hierarchical learning strategy in semantic relation extraction

A Minimum Description Length Approach to Multitask Feature Selection

Semi-supervised Feature Analysis by Mining Correlations among Multiple Tasks

Classifying Relations Via Long Short Term Memory Networks along Shortest Dependency Paths

Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling

Commonly Uncommon: Semantic Sparsity in Situation Recognition

Multi-task Pre-training Language Model for Semantic Network Completion

A Unified Architecture for Semantic Role Labeling and Relation Classification

Multi-view scene matching with relation aware feature perception