Abstract:Knowledge graph completion (KGC) tasks aim to infer missing facts in a knowledge graph (KG) for many knowledge-intensive applications. However, existing embedding-based KGC approaches primarily rely on factual triples, potentially leading to outcomes inconsistent with common sense. Besides, generating explicit common sense is often impractical or costly for a KG. To address these challenges, we propose a pluggable common sense-enhanced KGC framework that incorporates both fact and common sense for KGC. This framework is adaptable to different KGs based on their entity concept richness and has the capability to automatically generate explicit or implicit common sense from factual triples. Furthermore, we introduce common sense-guided negative sampling and a coarse-to-fine inference approach for KGs with rich entity concepts. For KGs without concepts, we propose a dual scoring scheme involving a relation-aware concept embedding mechanism. Importantly, our approach can be integrated as a pluggable module for many knowledge graph embedding (KGE) models, facilitating joint common sense and fact-driven training and inference. The experiments illustrate that our framework exhibits good scalability and outperforms existing models across various KGC tasks.
What problem does this paper attempt to address?
This paper attempts to solve three main problems in the Knowledge Graph Completion (KGC) task:
1. **Lack of ready - made common sense**: Existing embedding - based knowledge graph completion methods mainly rely on fact triples, which may lead to results that do not conform to common sense. Common sense is represented in the form of concepts and their relationships in some popular common - sense knowledge graphs, but the concepts in these common - sense knowledge graphs are usually not linked to entities and cannot be directly applied to entity - centered KGC tasks.
2. **Ineffective negative sampling**: In the training stage, most KGE models adopt negative sampling strategies to generate negative samples, but these strategies may generate low - quality or wrong negative samples, thus affecting the model performance. For example, low - quality negative samples (such as (San Francisco, LocatedIn, Iphone)) have too large a semantic gap from positive samples, resulting in ineffective training.
3. **Uncertainty in the reasoning stage**: In the reasoning stage, due to the uncertainty of KG embedding, incorrect entities may be ranked ahead of correct entities. For example, in the example shown in Figure 1, although the tail entity California predicted by the KGE model has the highest score, its corresponding concept State does not conform to the common sense (Person, Nationality, Country).
To solve these problems, the author proposes a plug - in common - sense - enhanced framework, which has the following characteristics:
- **Automatic common - sense generation**: For knowledge graphs containing rich entity concepts, explicit common sense can be automatically generated; for knowledge graphs lacking concepts, implicit common sense can be generated by expanding fact triples.
- **Common - sense - guided high - quality negative sampling**: A common - sense - guided negative sampling strategy is designed to ensure that the generated negative samples are both high - quality and diverse.
- **Coarse - to - fine reasoning mechanism**: A coarse - to - fine reasoning mechanism is proposed to ensure that the predicted triples conform to common sense.
Through these improvements, this framework not only improves the accuracy of the KGC task, but also can be integrated as a plug - in module into existing KGE models to achieve common - sense - and fact - driven joint training and reasoning.
### Formula Summary
1. **Translation Model Scoring Function (TransE)**:
\[
E_{ec}(h, r, t)=-\|h + r - t\|
\]
where \(h, r, t\) are the vector embeddings of the head entity, the relation, and the tail entity respectively.
2. **Rotation Model Scoring Function (RotatE)**:
\[
E_{ec}(h, r, t)=-\|h\circ r - t\|
\]
where \(\circ\) represents the Hadamard product, and \(h, r, t\) are vector embeddings in the complex space.
3. **Tensor Decomposition Model Scoring Function (ComplEx)**:
\[
E_{ec}(h, r, t)=\text{Re}(h^{\top}\text{diag}(r)t)
\]
where \(\text{diag}(r)\) is the diagonal matrix corresponding to \(r\) in the complex space, \(h\) and \(t\) are complex vectors, and \(t\) represents the conjugate of \(t\).
4. **Negative Sample Weight Calculation**:
\[
w(h'_j, r, t)=1 - p(h'_j, r, t)=1-\frac{\exp(E_{ec}(h'_j, r, t))}{\sum_{i}\exp(E_{ec}(h'_i, r, t))}
\]
\[
w(h, r, t'_j)=p(h, r, t'_j)=\frac{\exp(E_{ec}(h, r, t'_j))}{\sum_{i}\exp(E_{ec}(h, r, t'_i))}
\]
5. **Loss Function**:
\[
L = \sum_{i}