Abstract:In network representation learning we learn how to represent heterogeneous information networks in a low-dimensional space so as to facilitate effective search, classification, and prediction solutions. Previous network representation learning methods typically require sufficient task-specific labeled data to address domain-specific problems. The trained model usually cannot be transferred to out-of-domain datasets. We propose a self-supervised pre-training and fine-tuning framework, <inline-formula><tex-math notation="LaTeX">$\mathsf{PF\text{-}HIN}$</tex-math></inline-formula> , to capture the features of a heterogeneous information network. Unlike traditional network representation learning models that have to train the entire model all over again for every downstream task and dataset, <inline-formula><tex-math notation="LaTeX">$\mathsf{PF\text{-}HIN}$</tex-math></inline-formula> only needs to fine-tune the model and a small number of extra task-specific parameters, thus improving model efficiency and effectiveness. During pre-training, we first transform the neighborhood of a given node into a sequence. <inline-formula><tex-math notation="LaTeX">$\mathsf{PF\text{-}HIN}$</tex-math></inline-formula> is pre-trained based on two self-supervised tasks, masked node modeling and adjacent node prediction. We adopt deep bi-directional transformer encoders to train the model, and leverage factorized embedding parameterization and cross-layer parameter sharing to reduce the parameters. In the fine-tuning stage, we choose four benchmark downstream tasks, i.e., link prediction, similarity search, node classification, and node clustering. <inline-formula><tex-math notation="LaTeX">$\mathsf{PF\text{-}HIN}$</tex-math></inline-formula> outperforms state-of-the-art alternatives on each of these tasks, on four datasets.

A Tensor-based Markov Chain Model for Heterogeneous Information Network Collective Classification

A Noval Weighted Meta Graph Method for Classification in Heterogeneous Information Networks

Meta Path-Based Collective Classification in Heterogeneous Information Networks

Second-Order Markov Assumption Based Bayes Classifier for Networked Data with Heterophily

A Random Walk Tensor Model for Heterogeneous Network Entity Classification

Deep Collective Classification in Heterogeneous Information Networks

CHIN: Classification with META-PATH in Heterogeneous Information Networks.

A New Hybrid Collective Classification Method Based on Random Walk and Link Pattern.

Multi-label Collective Classification Via Markov Chain Based Learning Method

Multi-semantic Metapath Based Classification Method in Heterogeneous Information Network

H<SUP>2</SUP>TNE: Temporal Heterogeneous Information Network Embedding in Hyperbolic Spaces

(HTNE)-T-2: Temporal Heterogeneous Information Network Embedding in Hyperbolic Spaces

Higher-Order Clustering in Heterogeneous Information Networks.

Multi-type Co-clustering of General Heterogeneous Information Networks via Nonnegative Matrix Tri-Factorization

<inline-formula><tex-math notation="LaTeX">$\mathsf{PF\text{-}HIN}$</tex-math></inline-formula>:Pre-Training for Heterogeneous Information Networks

LIHAN: A Lattice-Guided Incomplete Heterogeneous Information Network Embedding Model for Node Classification

Key Nodes Cluster Augmented Embedding for Heterogeneous Information Networks.

Leveraging Meta-path Contexts for Classification in Heterogeneous Information Networks

Nonnegative Matrix Tri-Factorization Based Clustering in a Heterogeneous Information Network with Star Network Schema

$\text{H}^2\text{TNE}$: Temporal Heterogeneous Information Network Embedding in Hyperbolic Spaces