Abstract:Knowledge Graph Completion (KGC), aiming to infer the missing part of Knowledge Graphs (KGs), has long been treated as a crucial task to support downstream applications of KGs, especially for the multimodal KGs (MKGs) which suffer the incomplete relations due to the insufficient accumulation of multimodal corpus. Though a few research attentions have been paid to the completion task of MKGs, there is still a lack of specially designed negative sampling strategies tailored to MKGs. Meanwhile, though effective negative sampling strategies have been widely regarded as a crucial solution for KGC to alleviate the vanishing gradient problem, we realize that, there is a unique challenge for negative sampling in MKGs about how to model the effect of KG relations during learning the complementary semantics among multiple modalities as an extra context. In this case, traditional negative sampling techniques which only consider the structural knowledge may fail to deal with the multimodal KGC task. To that end, in this paper, we propose a MultiModal Relation-enhanced Negative Sampling (MM-RNS) framework for multimodal KGC task. Especially, we design a novel knowledge-guided cross-modal attention (KCA) mechanism, which provides bi-directional attention for visual & textual features via integrating relation embedding. Then, an effective contrastive semantic sampler is devised after consolidating the KCA mechanism with contrastive learning. In this way, a more similar representation of semantic features between positive samples, as well as a more diverse representation between negative samples under different relations could be learned. Afterwards, a masked gumbel-softmax optimization mechanism is utilized for solving the non-differentiability of sampling process, which provides effective parameter optimization compared with traditional sample strategies. Extensive experiments on three multimodal KGs demonstrate that our MMRNS framework could significantly outperform the state-of-the-art baseline methods, which validates the effectiveness of relation guides in multimodal KGC task.

Relation-enhanced Negative Sampling for Multimodal Knowledge Graph Completion

Mixed Geometry Message and Trainable Convolutional Attention Network for Knowledge Graph Completion

MACO: A Modality Adversarial and Contrastive Framework for Modality-missing Multi-modal Knowledge Graph Completion

Modality-Aware Negative Sampling for Multi-modal Knowledge Graph Embedding

Knowledge Graph Completion with Pre-trained Multimodal Transformer and Twins Negative Sampling

Relation Adversarial Network for Low Resource Knowledge Graph Completion

Relation-Aware Multi-Positive Contrastive Knowledge Graph Completion with Embedding Dimension Scaling

Unified Interpretation of Smoothing Methods for Negative Sampling Loss Functions in Knowledge Graph Embedding

NativE: Multi-modal Knowledge Graph Completion in the Wild

Contrast then Memorize: Semantic Neighbor Retrieval-Enhanced Inductive Multimodal Knowledge Graph Completion

SCI-MKGC: A MKGC Method Based on Spatial Context and Interaction Attention

MoSE: Modality Split and Ensemble for Multimodal Knowledge Graph Completion

Multimodal Biological Knowledge Graph Completion via Triple Co-Attention Mechanism.

KERMIT: Knowledge Graph Completion of Enhanced Relation Modeling with Inverse Transformation

KG-NSF: Knowledge Graph Completion with a Negative-Sample-Free Approach

Negative Sampling with Adaptive Denoising Mixup for Knowledge Graph Embedding

Relation Extraction with Knowledge-Enhanced Prompt-Tuning on Multimodal Knowledge Graph

Semantic-Enhanced Knowledge Graph Completion

Boosting BERT-Based Knowledge Graph Completion with Contrastive Learning and Hard Sample Training

Entity Similarity-Based Negative Sampling for Knowledge Graph Embedding

Simplified Multi-View Graph Neural Network for Multilingual Knowledge Graph Completion