Abstract:Learning vector representations (i.e., embeddings) of nodes for graph-structured information network has attracted vast interest from both industry and academia. Most real-world networks exhibit a complex and heterogeneous format, enclosing high-order relationships and rich semantic information among nodes. However, existing heterogeneous network embedding (HNE) frameworks are commonly designed in a centralized fashion, i.e., all the data storage and learning process take place on a single machine. Hence, those HNE methods show severe performance bottlenecks when handling large-scale networks due to high consumption on memory, storage, and running time. In light of this, to cope with large-scale HNE tasks with strong efficiency and effectiveness guarantee, we propose Decentralized Deep Heterogeneous Hypergraph (DDHH) embedding framework in this paper. In DDHH, we innovatively formulate a large heterogeneous network as a hypergraph, where its hyperedges can connect a set of semantically similar nodes. Our framework then intelligently partitions the heterogeneous network using the identified hyperedges. Then, each resulted subnetwork is assigned to a distributed worker, which employs the deep information maximization theorem to locally learn node embeddings from the partition received. We further devise a novel embedding alignment scheme to precisely project independently learned node embeddings from all subnetworks onto a public vector space, thus allowing for downstream tasks. As shown from our experimental results, DDHH significantly improves the efficiency and accuracy of existing HNE models, and can easily scale up to large-scale heterogeneous networks.

Transferable and Differentiable Discrete Network Embedding for multi-domains with Hierarchical Knowledge Distillation

Transfer Learning and Deep Domain Adaptation

Generalized Deep Transfer Networks for Knowledge Propagation in Heterogeneous Domains.

Deep Model Transferability from Attribution Maps

Collaborative Knowledge Distillation for Heterogeneous Information Network Embedding

Weakly-Shared Deep Transfer Networks For Heterogeneous-Domain Knowledge Propagation

Semi-supervised Network Embedding with Differentiable Deep Quantisation

Hierarchical Embedding Methods for Large-scale Complex Knowledge Graphs

Semisupervised Network Embedding With Differentiable Deep Quantization

Direct Distillation between Different Domains

Harmonizing knowledge Transfer in Neural Network with Unified Distillation

Deep Transfer Neural Network Using Hybrid Representations of Domain Discrepancy

Distributed Representation Learning of Knowledge Graph with Diverse Information

Active Domain Transfer on Network Embedding.

DDHH: A Decentralized Deep Learning Framework for Large-scale Heterogeneous Networks

Disentangling Transfer and Interference in Multi-Domain Learning

Graph Enabled Cross-Domain Knowledge Transfer

Unsupervised Domain-adaptive Hash for Networks

Distilling Holistic Knowledge with Graph Neural Networks

Graph Embedding-Based Heterogeneous Domain Adaptation with Domain-Invariant Feature Learning and Distributional Order Preserving.

Embedded Mutual Learning: A Novel Online Distillation Method Integrating Diverse Knowledge Sources.