Hierarchical Message-Passing Graph Neural Networks

Zhiqiang Zhong,Cheng-Te Li,Jun Pang

DOI: https://doi.org/10.48550/arXiv.2009.03717

2022-10-26

Abstract:Graph Neural Networks (GNNs) have become a prominent approach to machine learning with graphs and have been increasingly applied in a multitude of domains. Nevertheless, since most existing GNN models are based on flat message-passing mechanisms, two limitations need to be tackled: (i) they are costly in encoding long-range information spanning the graph structure; (ii) they are failing to encode features in the high-order neighbourhood in the graphs as they only perform information aggregation across the observed edges in the original graph. To deal with these two issues, we propose a novel Hierarchical Message-passing Graph Neural Networks framework. The key idea is generating a hierarchical structure that re-organises all nodes in a flat graph into multi-level super graphs, along with innovative intra- and inter-level propagation manners. The derived hierarchy creates shortcuts connecting far-away nodes so that informative long-range interactions can be efficiently accessed via message passing and incorporates meso- and macro-level semantics into the learned node representations. We present the first model to implement this framework, termed Hierarchical Community-aware Graph Neural Network (HC-GNN), with the assistance of a hierarchical community detection algorithm. The theoretical analysis illustrates HC-GNN's remarkable capacity in capturing long-range information without introducing heavy additional computation complexity. Empirical experiments conducted on 9 datasets under transductive, inductive, and few-shot settings exhibit that HC-GNN can outperform state-of-the-art GNN models in network analysis tasks, including node classification, link prediction, and community detection. Moreover, the model analysis further demonstrates HC-GNN's robustness facing graph sparsity and the flexibility in incorporating different GNN encoders.

Machine Learning

What problem does this paper attempt to address?

The problems that this paper attempts to solve mainly focus on two key limitations in Graph Neural Networks (GNNs):

1. **High cost of long - distance information encoding**: Most existing GNN models are based on a flat message - passing mechanism, which makes them costly when encoding long - distance information across the graph structure.
2. **Inability to encode high - order neighborhood features**: These models can only aggregate information on the edges observed in the original graph, so it is difficult to capture high - order neighborhood features in the graph.

To overcome these two problems, the authors propose a new framework - **Hierarchical Message - Passing Graph Neural Networks (HMGNNs)**. The core idea of this framework is to generate a hierarchical structure, reorganize all nodes in the graph into multi - level super - graphs, and introduce innovative intra - layer and inter - layer propagation methods. Specifically:

- **Generation of hierarchical structure**: Gradually simplify the input graph \(G\) by grouping nodes into super - nodes at different levels.
- **Construction of hierarchical super - graphs**: Construct the super - graph \(G_t\) of the \(t\) - th layer according to the connection relationships of nodes in the graph \(G_{t - 1}\) of the \((t - 1)\) - th layer.
- **Hierarchical message passing**: Develop three different propagation schemes, including bottom - up, within - level, and top - down propagation, to achieve cross - level message passing.
- **Model learning**: Use task - related loss functions and the gradient descent process to train the model.

Through these methods, HMGNNs can effectively capture long - distance information and incorporate meso - and macro - level semantics into node representations, thus performing well in various network analysis tasks, including node classification, link prediction, and community detection. Experimental results show that HC - GNN (a specific implementation of this framework) significantly outperforms existing GNN models on multiple datasets.