Knowledge Probing for Graph Representation Learning

Mingyu Zhao,Xingyu Huang,Ziyu Lyu,Yanlin Wang,Lixin Cui,Lu Bai

2024-08-08

Abstract:Graph learning methods have been extensively applied in diverse application areas. However, what kind of inherent graph properties e.g. graph proximity, graph structural information has been encoded into graph representation learning for downstream tasks is still under-explored. In this paper, we propose a novel graph probing framework (GraphProbe) to investigate and interpret whether the family of graph learning methods has encoded different levels of knowledge in graph representation learning. Based on the intrinsic properties of graphs, we design three probes to systematically investigate the graph representation learning process from different perspectives, respectively the node-wise level, the path-wise level, and the structural level. We construct a thorough evaluation benchmark with nine representative graph learning methods from random walk based approaches, basic graph neural networks and self-supervised graph methods, and probe them on six benchmark datasets for node classification, link prediction and graph classification. The experimental evaluation verify that GraphProbe can estimate the capability of graph representation learning. Remaking results have been concluded: GCN and WeightedGCN methods are relatively versatile methods achieving better results with respect to different tasks.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: currently, the research on graph representation learning methods in encoding the inherent properties of graphs (such as graph proximity, graph structure information, etc.) is still insufficient. Although graph learning methods have been widely applied in multiple fields, there is a lack of systematic evaluation to explore whether and how these methods encode different levels of knowledge in graph representation learning. Therefore, the author proposes a novel graph probing framework (GraphProbe), aiming to systematically investigate and explain whether and how graph learning methods encode different levels of knowledge in graph representation learning from three different perspectives: node - level, path - level, and structure - level.

Specifically, the main objectives of the paper include:
1. **Propose a systematic graph probing framework**: By designing three detectors (node centrality detector, distance detector, and structure detector), respectively evaluate the capabilities of graph representation learning methods from three aspects: node - level, path - level, and structure - level.
2. **Construct an evaluation benchmark**: Use nine representative graph learning methods (including random - walk - based methods, basic graph neural networks, self - supervised graph learning methods, and weighted graph learning methods) to evaluate on six benchmark datasets, covering three classic downstream tasks: node classification, link prediction, and graph classification.
3. **Verify the effectiveness of the probing framework**: Through experimental verification, GraphProbe can effectively evaluate the capabilities of graph representation learning methods, and its results are competitive and consistent with traditional evaluation metrics (such as accuracy, AUC, and F1 - score).

In conclusion, this paper fills the gap in the systematic evaluation of graph representation learning methods in knowledge encoding by proposing the GraphProbe framework, providing a new perspective for understanding the performance of different graph learning methods in different tasks.