Abstract:<p>Knowledge graph (KG) embedding projects the graph into a low-dimensional space and preserves the graph information. An essential part of a KG is the ontology, which always is organized as a taxonomy tree, depicting the type (or multiple types) of each entity and the hierarchical relationships among these types. The importance of considering the ontology during KG embedding lies in its ability to provide side-information, improving the downstream applications' accuracy (e.g., link prediction, entity alignment or recommendation). However, the ontology has yet to receive adequate attention during the KG embedding, especially for instances where each entity may belong to multiple types. This ontology-enhanced KG embedding's main challenges are two-fold: determining how to discover the relationships among these types and how to integrate them with the entities' relationship network. Although it is common to see attention-based models used in KG embedding, they cannot settle the issues raised simultaneously. Only a single type is assigned to each entity and the correlation among types are ignored in those models, leading to information loss and encumbered downstream tasks. To overcome these challenges, we propose a composite multi-type aware KG embedding model, whose main components are a multi-type layer and entity embedding layer. We model it as a natural language processing task at the multi-type layer to discover each entity's multi-type feature and automatically capture their correlations. Additionally, a relation-based attention mechanism is conducted at the entity embedding layer, which aggregates neighborhoods' information and integrates the multi-type layer's information through common entities of these two layers. Through extensive experiments on two real KGs, we demonstrate that, compared to several state-of-the-art baselines, our Multi-Type aware Embedding (MTE) model achieves substantial gain in both Mean Rank and for the link prediction task and accuracy for multi-type classification.</p>

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the deficiency of existing knowledge graph (KG) embedding methods in dealing with multi - type information of entities. Specifically, most of the existing KG embedding methods only consider that each entity has one type and ignore the associations between types, which leads to information loss and affects the accuracy of downstream tasks (such as link prediction, entity alignment or recommendation). The paper proposes a new composite multi - type - aware KG embedding model (MTE), aiming to overcome these challenges. ### Specific problems solved by the paper: 1. **Processing of multi - type entities**: - Existing KG embedding methods usually assume that each entity has only one type, while in fact many entities may belong to multiple types. For example, the movie director Steven Spielberg can be "director", "producer" and "male" at the same time. This multi - type information is crucial for improving the effect of KG embedding. 2. **Associations between types**: - There are complex association relationships between entity types, and these relationships can be represented by a taxonomy tree. For example, "male" and "female" both belong to "human", while "male" and "dog" have no direct association. Existing methods often ignore these associations between types, resulting in information loss. 3. **Fusion of relations and types**: - The relation and type information between entities are interrelated. For example, the roles of an entity as "director" and "producer" can be inferred through its relations with neighboring entities. Existing methods usually cannot effectively combine these two kinds of information, thus affecting the performance of the model. ### Solutions: The paper proposes a two - layer composite model MTE, specifically including: 1. **Multi - type layer**: - This layer embeds the multi - type information of entities into a vector and models the correlations between types through common entities and the taxonomy tree. In this way, it can effectively capture the multi - type features of each entity and the associations between their types. 2. **Entity embedding layer**: - This layer uses a relation - based attention mechanism to aggregate the information of neighboring entities and transfer the information of the multi - type layer through common entities. In this way, it can not only learn the local features of each entity, but also learn the importance of different types of relations and assign appropriate weights to them. ### Main contributions: 1. **Learning multi - type - aware entity embeddings based on GNN for the first time**: - By utilizing the relationships between types instead of simply regarding types as text information, the quality of entity embeddings is improved. 2. **Designing a two - layer composite model**: - The first layer learns the multi - type feature embeddings of entities, and the second layer combines the initial type information and relation embeddings, taking into account hierarchical multi - types and relation diversity at the same time. 3. **Proposing a relation - and hierarchical - multi - type - aware attention mechanism**: - A two - level attention mechanism is designed. The first level calculates the contributions of neighboring nodes within the same relation group, and the second level calculates the weights of different relations, so as to perform information aggregation more effectively. 4. **Experimental verification**: - Extensive experiments were carried out on two common KG datasets, and the results show that the MTE model is significantly superior to the existing baseline models in link prediction and multi - label classification tasks. Through these innovations, the paper provides a more comprehensive and effective KG embedding method, especially in dealing with multi - type information of entities, and provides a solid foundation for subsequent KG applications.

Relation-based multi-type aware knowledge graph embedding

KTAT: A Complex Embedding Model of Knowledge Graph Integrating Type Information and Attention Mechanism

AutoETER: Automated Entity Type Representation with Relation-Aware Attention for Knowledge Graph Embedding

Meta-Knowledge Transfer for Inductive Knowledge Graph Embedding

Connecting Embeddings Based on Multiplex Relational Graph Attention Networks for Knowledge Graph Entity Typing

Knowledge Graph Embedding with Hierarchical Relation Structure

Multi-perspective knowledge graph completion with global and interaction features

AutoETER: Automated Entity Type Representation for Knowledge Graph Embedding.

A knowledge graph embedding model based attention mechanism for enhanced node information integration

Knowledge Graph Embedding: A Survey of Approaches and Applications

Contrastive Multi-modal Knowledge Graph Representation Learning

Multi-view Knowledge Graph Embedding for Entity Alignment

Global Graph Attention Embedding Network for Relation Prediction in Knowledge Graphs

Category-Embodied Knowledge Embedding.

METransE: Manifold-like Mechanism Enhanced Embedding for Reasoning over Knowledge Graphs

Knowledge Graph Embedding Based on Multi-View Clustering Framework

Knowledge Graph Embedding Via Reasoning over Entities, Relations, and Text

Heuristic-Driven, Type-Specific Embedding in Parallel Spaces for Enhancing Knowledge Graph Reasoning.

Pre-train Unified Knowledge Graph Embedding with Ontology

Learning graph attention-aware knowledge graph embedding