Abstract:Recently, neural network based methods have shown their power in learning more expressive features on the task of knowledge graph embedding (KGE). However, the performance of deep methods often falls behind the shallow ones on simple graphs. One possible reason is that deep models are difficult to train, while shallow models might suffice for accurately representing the structure of the simple KGs. In this paper, we propose a neural network based model, named DeepE, to address the problem, which stacks multiple building blocks to predict the tail entity based on the head entity and the relation. Each building block is an addition of a linear and a non-linear function. The stacked building blocks are equivalent to a group of learning functions with different non-linear depth. Hence, DeepE allows deep functions to learn deep features, and shallow functions to learn shallow features. Through extensive experiments, we find DeepE outperforms other state-of-the-art baseline methods. A major advantage of DeepE is the robustness. DeepE achieves a Mean Rank (MR) score that is 6%, 30%, 65% lower than the best baseline methods on FB15k-237, WN18RR and YAGO3-10. Our design makes it possible to train much deeper networks on KGE, e.g. 40 layers on FB15k-237, and without scarifying precision on simple relations.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the poor performance of existing deep - learning models on simple Knowledge Graphs (KGs). Specifically: 1. **Difficulties in training deep models**: Although deep neural networks have advantages on complex graphs, they are difficult to train on simple graphs and are prone to over - fitting. 2. **Limitations of shallow models**: Shallow models can represent the structure of simple graphs well, but they cannot capture deep - level features. 3. **The need for co - existence of shallow and deep features**: The relationships in knowledge graphs include both simple linear relationships (such as symmetry, inversion, etc.) and complex non - linear relationships. Therefore, a model that can learn both shallow and deep features is required. To solve these problems, the authors propose the DeepE model, which simultaneously learns shallow and deep features by stacking multiple building blocks. Each building block contains linear and non - linear parts, enabling the model to flexibly switch between learning functions at different depths, thus performing well on both simple and complex graphs. ### Key features of the DeepE model - **Mixed - learning functions**: DeepE forms learning functions with different non - linear depths by stacking multiple building blocks. This enables the model to handle shallow and deep features simultaneously. - **Robustness**: DeepE shows stronger robustness in cases of sparse data or complex relationships. Even if some sub - functions are not applicable, other sub - functions can still work normally. - **The ability to train deep networks**: DeepE can train very deep networks (e.g., 40 - layer networks) while maintaining the accuracy of simple relationships, which is difficult to achieve with existing methods. ### Experimental verification Through experiments on multiple benchmark datasets (such as FB15k - 237, WN18RR, and YAGO3 - 10), DeepE has achieved results significantly better than existing methods in terms of Mean Rank (MR), Mean Reciprocal Rank (MRR), Hit@1, and Hit@10. In particular, there is a large improvement in the MR metric, indicating its advantage in terms of robustness. In conclusion, DeepE solves the problem of poor performance of deep models on simple graphs through innovative design and demonstrates excellent performance and robustness in various situations.

DeepE: a deep neural network for knowledge graph embedding

Knowledge Graph Embedding with Entity Neighbors and Deep Memory Network

From Wide to Deep: Dimension Lifting Network for Parameter-efficient Knowledge Graph Embedding

A Knowledge Graph Embedding Method Based on Neural Network

Multi-scale Dilated Convolutional Network for Knowledge Graph Embedding

RIECN: Learning Relation-Based Interactive Embedding Convolutional Network for Knowledge Graph.

ShallowBKGC: a BERT-enhanced shallow neural network model for knowledge graph completion

Deep Learning in Knowledge Graph

Meta-Knowledge Transfer for Inductive Knowledge Graph Embedding

NeuralKG: an Open Source Library for Diverse Representation Learning of Knowledge Graphs

Multi-Scale Dynamic Convolutional Network for Knowledge Graph Embedding

A data-centric framework of improving graph neural networks for knowledge graph embedding

Convolutional 2D Knowledge Graph Embeddings

RelaGraph: Improving Embedding on Small-Scale Sparse Knowledge Graphs by Neighborhood Relations

Knowledge graph embedding model with attention-based high-low level features interaction convolutional network

DSparsE: Dynamic Sparse Embedding for Knowledge Graph Completion

A knowledge graph embedding model based attention mechanism for enhanced node information integration

Evaluating Deep Graph Neural Networks

Knowledge graph embedding based on dynamic adaptive atrous convolution and attention mechanism for link prediction

Improving Knowledge Graph Embedding Using Dynamic Aggregation of Neighbor Information.