Abstract:Knowledge representation learning has received a lot of attention in the past few years. The success of existing methods heavily relies on the quality of knowledge graphs. The entities with few triplets tend to be learned with less expressive power. Fortunately, there are many knowledge graphs constructed from various sources, the representations of which could contain much information. We propose an adversarial embedding transfer network ATransN, which transfers knowledge from one or more teacher knowledge graphs to a target one through an aligned entity set without explicit data leakage. Specifically, we add soft constraints on aligned entity pairs and neighbours to the existing knowledge representation learning methods. To handle the problem of possible distribution differences between teacher and target knowledge graphs, we introduce an adversarial adaption module. The discriminator of this module evaluates the degree of consistency between the embeddings of an aligned entity pair. The consistency score is then used as the weights of soft constraints. It is not necessary to acquire the relations and triplets in teacher knowledge graphs because we only utilize the entity representations. Knowledge graph completion results show that ATransN achieves better performance against baselines without transfer on three datasets, CN3l, WK3l, and DWY100k. The ablation study demonstrates that ATransN can bring steady and consistent improvement in different settings. The extension of combining other knowledge graph embedding algorithms and the extension with three teacher graphs display the promising generalization of the adversarial transfer network.

Attribution Guided Layerwise Knowledge Amalgamation from Graph Neural Networks

Knowledge Distillation Improves Graph Structure Augmentation for Graph Neural Networks

Amalgamating Knowledge from Heterogeneous Graph Neural Networks

Mixed Geometry Message and Trainable Convolutional Attention Network for Knowledge Graph Completion

Hierarchical Knowledge Amalgamation with Dual Discriminative Feature Alignment

Online Cross-Layer Knowledge Distillation on Graph Neural Networks with Deep Supervision

Customizing Student Networks From Heterogeneous Teachers Via Adaptive Knowledge Amalgamation

Alignahead: Online Cross-Layer Knowledge Extraction on Graph Neural Networks

Collaborative knowledge amalgamation: Preserving discriminability and transferability in unsupervised learning

Amalgamating Knowledge towards Comprehensive Classification

Knowledge Amalgamation from Heterogeneous Networks by Common Feature Learning

Compressing Deep Graph Neural Networks via Adversarial Knowledge Distillation

Contrastive Knowledge Amalgamation for Unsupervised Image Classification

On Representation Knowledge Distillation for Graph Neural Networks

Frameless Graph Knowledge Distillation

ALP-KD: Attention-Based Layer Projection for Knowledge Distillation

Online Adversarial Knowledge Distillation for Graph Neural Networks

An Adversarial Transfer Network for Knowledge Representation Learning

Boosting Graph Neural Networks via Adaptive Knowledge Distillation

Fine-Grained Learning Behavior-Oriented Knowledge Distillation for Graph Neural Networks

Teaching MLP More Graph Information: A Three-stage Multitask Knowledge Distillation Framework