Abstract:Entity representation learning is both fundamental and crucial for link prediction in knowledge graph. Existing entity representation learning approaches mainly focus on learning from knowledge triples with single modal text data which could only describe the things or concepts in words. However, multimodal data commonly exists in the real worlds and may describe things or concepts from different aspects. Therefore, multimodal entities have emerged in knowledge graph in recent years, which greatly enrich the knowledge graph by different information. This paper focuses on how to learn more robust and comprehensive representations of multimodal entities for better link prediction. A new representation learning method for multimodal entities is proposed, called Graph Attention with Bi-pooling Representation Learning (GBRL), where multimodal fusion and knowledge embedding are combined for learning representations of an entity with consideration of its multimodal neighbors. Firstly, feature vectors are obtained by pre-trained models for each modality. Then, the bilinear pooling is utilized for combining feature vectors of multimodal entities into multimodal feature vectors. A two-layer graph attention module is followed to generate multimodal representations of an entity from its multimodal neighbors. Finally, the inception decoder uses multimodal knowledge embedding to calculate scores for link prediction. The experimental results have shown that the proposed method performs well on link prediction by consideration of multimodal entities comparing with traditional methods that using only single modal text data.

Multi-modal knowledge graphs representation learning via multi-headed self-attention

Knowledge Representation Learning with Entity Descriptions, Hierarchical Types, and Textual Relations

Contrastive Multi-modal Knowledge Graph Representation Learning

Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey

Entity Representation Learning with Multimodal Neighbors for Link Prediction in Knowledge Graph

The Power of Noise: Toward a Unified Multi-modal Knowledge Graph Representation Framework.

Multi-modal Recommendation Based on Knowledge Graph

Enhancing Recommender System with Multi-modal Knowledge Graph.

Multi-modal Knowledge Graphs for Recommender Systems

MMKG: Multi-modal Knowledge Graphs

Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learning

A Model of Text-Enhanced Knowledge Graph Representation Learning with Mutual Attention.

Noise-powered Multi-modal Knowledge Graph Representation Framework

An attention‐based representation learning model for multiple relational knowledge graph

Representation Learning over Multiple Knowledge Graphs for Knowledge Graphs Alignment

MMEA: Entity Alignment for Multi-modal Knowledge Graph.

Multi-modal Siamese Network for Entity Alignment

A Multi -Role Graph Attention Network for Knowledge Graph Alignment.

Multi-modal Multi-relational Feature Aggregation Network for Medical Knowledge Representation Learning

A Survey of Multi-modal Knowledge Graphs: Technologies and Trends

Self-supervised opinion summarization with multi-modal knowledge graph