Simple and Efficient Heterogeneous Graph Neural Network

Xiaocheng Yang,Mingyu Yan,Shirui Pan,Xiaochun Ye,Dongrui Fan

2023-09-01

Abstract:Heterogeneous graph neural networks (HGNNs) have powerful capability to embed rich structural and semantic information of a heterogeneous graph into node representations. Existing HGNNs inherit many mechanisms from graph neural networks (GNNs) over homogeneous graphs, especially the attention mechanism and the multi-layer structure. These mechanisms bring excessive complexity, but seldom work studies whether they are really effective on heterogeneous graphs. This paper conducts an in-depth and detailed study of these mechanisms and proposes Simple and Efficient Heterogeneous Graph Neural Network (SeHGNN). To easily capture structural information, SeHGNN pre-computes the neighbor aggregation using a light-weight mean aggregator, which reduces complexity by removing overused neighbor attention and avoiding repeated neighbor aggregation in every training epoch. To better utilize semantic information, SeHGNN adopts the single-layer structure with long metapaths to extend the receptive field, as well as a transformer-based semantic fusion module to fuse features from different metapaths. As a result, SeHGNN exhibits the characteristics of simple network structure, high prediction accuracy, and fast training speed. Extensive experiments on five real-world heterogeneous graphs demonstrate the superiority of SeHGNN over the state-of-the-arts on both accuracy and training speed.

Machine Learning

What problem does this paper attempt to address?

The paper aims to address the limitations and inefficiencies present in existing heterogeneous graph neural networks (HGNNs) by proposing a new model called Simple and Efficient Heterogeneous Graph Neural Network (SeHGNN). The primary problems identified in current HGNNs include:

1. **Inheritance of Complex Mechanisms from Homogeneous GNNs**: Many HGNNs adopt mechanisms from graph neural networks (GNNs) designed for homogeneous graphs, such as the attention mechanism and multi-layer structures. However, the effectiveness of these mechanisms on heterogeneous graphs is not well-studied.

2. **Excessive Complexity and Computation**: The use of attention mechanisms and multi-layer structures in HGNNs leads to increased computational complexity and training time, which becomes a bottleneck for applying these models on large-scale heterogeneous graphs.

To tackle these issues, the paper conducts an in-depth analysis and makes two key findings:

1. **Finding 1**: Semantic attention is essential, while neighbor attention is not necessary. Semantic attention helps weigh the importance of different semantic relationships, whereas neighbor attention does not contribute significantly to the model's performance.

2. **Finding 2**: Models with a single-layer structure and long metapaths outperform those with multi-layer structures and short metapaths. Longer metapaths allow for better distinction between high-level semantic relationships, enhancing the model's performance.

Based on these