Abstract:The task of inductive link prediction in knowledge graphs (KGs) generally focuses on test predictions with solely new nodes but not both new nodes and new relation types. In this work, we formally define the concept of double permutation-equivariant representations that are equivariant to permutations of both node identities and edge relation types. We then show how double-equivariant architectures are able to self-supervise pre-train on distinct KG domains and zero-shot predict links on a new KG domain (with completely new entities and new relation types). We also introduce the concept of distributionally double equivariant positional embeddings designed to perform the same task. Finally, we empirically demonstrate the capability of the proposed models against baselines on a set of novel real-world benchmarks. More interestingly, we show that self-supervised pre-training on more KG domains increases the zero-shot ability of our model to predict on new relation types over new entities on unseen KG domains.
What problem does this paper attempt to address?
### What problem does this paper attempt to solve?
This paper aims to solve the dual - inductive link prediction task in knowledge graphs (KGs). Specifically, it not only focuses on the prediction of new nodes at test time, but also deals with the prediction of new relation types simultaneously. Traditional methods usually only predict new nodes and ignore the existence of new relation types. This paper proposes a new framework that can handle both situations, thereby achieving cross - domain zero - shot learning.
#### Main problem description
1. **Dual - inductive link prediction task**:
- Training phase: The model is trained on a set of known knowledge graphs.
- Testing phase: The model needs to predict missing links in completely unknown new knowledge graphs, which contain new nodes and new relation types that have never been seen before.
2. **Double equivariance**:
- Defines the permutation - equivariant representations of node identities and edge relation types.
- Proposes a double - equivariant architecture, enabling the model to perform self - supervised pre - training on knowledge graphs in different domains and zero - shot prediction in unseen new domains.
3. **Distributionally double - equivariant positional embeddings**:
- Introduces a new type of positional embedding method for performing the dual - inductive link prediction task.
- These embeddings are distributionally double - equivariant, ensuring the generalization ability of the model.
4. **Practical applications**:
- Experimental verification was carried out on two real - world benchmark datasets, PediaTypes and WikiTopics.
- Experiments show that as the number of self - supervised pre - trained KGs increases, the model's zero - shot prediction ability on new relation types also increases.
### Formulas and definitions
- **Knowledge graph representation**:
\[
A^{(\ast)} \in \{0, 1\}^{N^{(\ast)} \times R^{(\ast)} \times N^{(\ast)}}
\]
where \(N^{(\ast)}\) and \(R^{(\ast)}\) represent the number of nodes and the number of relation types respectively.
- **Definition of double equivariance**:
\[
\Gamma_{\text{gra}}(\phi \circ \tau \circ A^{(\ast)}) = \phi \circ \tau \circ \Gamma_{\text{gra}}(A^{(\ast)})
\]
where \(\phi \in S_{N^{(\ast)}}\) and \(\tau \in S_{R^{(\ast)}}\) represent the permutation operations of nodes and relations respectively.
- **Triple - double - invariant representation**:
\[
\Gamma_{\text{tri}}((i, k, j), A^{(\ast)}) = \Gamma_{\text{tri}}((\phi \circ i, \tau \circ k, \phi \circ j), \phi \circ \tau \circ A^{(\ast)})
\]
Through these formulas and definitions, this paper constructs a theoretical framework, enabling the model to perform effective zero - shot prediction in different knowledge graph domains.