Understanding Graph Neural Networks via Trajectory Analysis

Ziqiao Meng,Jin Dong,Zengfeng Huang,Irwin King
2019-01-01
Abstract:Graph neural network (GNN) has attracted enormous attentions in machine learning communities and analyzing the architecture of GNNs have been very popular these days. Inspired by Arora’s recent work, we propose utilizing "trajectory analysis" technique to see how the representation of each node evolves during training process by studying the differences of each node representation before and after a gradient update. Our studies in a simplified setting show that the update rule is quite simple and the rule indicates that the learning mechanism of GNNs could be divided into two stages: constructing similarity matrix and updating labels based on similarity matrix. Under semi-supervised learning settings, the node embedding udpate rule could even be regarded as weighted k-nearest neighbors algorithm. Last but not the least, convergence speed of different nodes will affect the model performance.
What problem does this paper attempt to address?