Knowledge Graph Completion Method Based on Two-level Attention to Aggregate Neighborhood Multimodal and Type Features

Li Xinrui,Zhang Xiaoming,Wang Huiyong
DOI: https://doi.org/10.1109/asip63198.2024.00030
2024-01-01
Abstract:The current research of knowledge graph completion usually considers various factors such as entity types and multimodal features. However, only considering entity types may blur the selection of entities due to the incomplete predefined entity types in the knowledge graph. Although multimodal features can enrich the semantics of entities, it may bring noise and redundant information affecting the representation effective. Therefore, we propose a knowledge graph completion method which comprehensively considers the entity types and text-image multimodal features to enhance the embedding representation of entities. Firstly, the features of triples, entity types and text-image are extracted respectively. Then, the structural representation with type constraints is obtained by adding the features of triples and types, and the multimodal fusion representation is obtained by weighted average of the features of text and image. Furtherly, we propose a two-level attention mechanism of relation and entity to aggregate entity neighborhood information including structural representation and multimodal representation, which can obtain more accurate and rich entity and relation embedding representations in different contexts. Experiments on both FB15K-237 and self-constructed domain dataset show that the proposed method outperforms the baselines.
What problem does this paper attempt to address?