Abstract:Knowledge representation learning has received a lot of attention in the past few years. The success of existing methods heavily relies on the quality of knowledge graphs. The entities with few triplets tend to be learned with less expressive power. Fortunately, there are many knowledge graphs constructed from various sources, the representations of which could contain much information. We propose an adversarial embedding transfer network ATransN, which transfers knowledge from one or more teacher knowledge graphs to a target one through an aligned entity set without explicit data leakage. Specifically, we add soft constraints on aligned entity pairs and neighbours to the existing knowledge representation learning methods. To handle the problem of possible distribution differences between teacher and target knowledge graphs, we introduce an adversarial adaption module. The discriminator of this module evaluates the degree of consistency between the embeddings of an aligned entity pair. The consistency score is then used as the weights of soft constraints. It is not necessary to acquire the relations and triplets in teacher knowledge graphs because we only utilize the entity representations. Knowledge graph completion results show that ATransN achieves better performance against baselines without transfer on three datasets, CN3l, WK3l, and DWY100k. The ablation study demonstrates that ATransN can bring steady and consistent improvement in different settings. The extension of combining other knowledge graph embedding algorithms and the extension with three teacher graphs display the promising generalization of the adversarial transfer network.

KTAN: Knowledge Transfer Adversarial Network.

An Adversarial Transfer Network for Knowledge Representation Learning

Adaptive Teaching with Shared Classifier for Knowledge Distillation

MKTN: Adversarial-Based Multifarious Knowledge Transfer Network from Complementary Teachers

Progressive Network Grafting for Few-Shot Knowledge Distillation

Adaptive Cross-Architecture Mutual Knowledge Distillation

Stage-by-stage Knowledge Distillation

Soft Hybrid Knowledge Distillation Against Deep Neural Networks

Online Adversarial Knowledge Distillation for Graph Neural Networks

An Embarrassingly Simple Approach for Knowledge Distillation

Attention and feature transfer based knowledge distillation

Improved Knowledge Distillation via Adversarial Collaboration

Common Knowledge Learning for Generating Transferable Adversarial Examples

Compressing Deep Graph Neural Networks via Adversarial Knowledge Distillation

Online Adversarial Distillation for Graph Neural Networks

Knowledge Distillation with Feature Maps for Image Classification

Knowledge Distillation Via Adaptive Meta-Learning for Graph Neural Network

Knowledge Augmentation for Distillation: A General and Effective Approach to Enhance Knowledge Distillation

Improving Knowledge Distillation With a Customized Teacher

Robust Knowledge Transfer Via Hybrid Forward on the Teacher-Student Model

Adaptive Informative Semantic Knowledge Transfer for Knowledge Distillation