Unleash Graph Neural Networks from Heavy Tuning

Lequan Lin,Dai Shi,Andi Han,Zhiyong Wang,Junbin Gao
2024-05-21
Abstract:Graph Neural Networks (GNNs) are deep-learning architectures designed for graph-type data, where understanding relationships among individual observations is crucial. However, achieving promising GNN performance, especially on unseen data, requires comprehensive hyperparameter tuning and meticulous training. Unfortunately, these processes come with high computational costs and significant human effort. Additionally, conventional searching algorithms such as grid search may result in overfitting on validation data, diminishing generalization accuracy. To tackle these challenges, we propose a graph conditional latent diffusion framework (GNN-Diff) to generate high-performing GNNs directly by learning from checkpoints saved during a light-tuning coarse search. Our method: (1) unleashes GNN training from heavy tuning and complex search space design; (2) produces GNN parameters that outperform those obtained through comprehensive grid search; and (3) establishes higher-quality generation for GNNs compared to diffusion frameworks designed for general neural networks.
Machine Learning
What problem does this paper attempt to address?
This paper focuses on reducing the heavy tuning problem of Graph Neural Networks (GNNs). GNNs are very useful in processing graph data, but they require a significant amount of hyperparameter tuning and fine-tuning to achieve good performance, which is time-consuming and requires a lot of human effort. Traditional grid search algorithms may lead to overfitting and affect generalization accuracy. To address these challenges, the paper proposes a graph conditional diffusion framework called GNN-Diff. This framework directly generates high-performance GNN parameters by a lightweight tuning coarse search based on saved checkpoints. GNN-Diff has three main advantages: 1. It reduces the reliance on complex tuning and search space design and provides a method for directly sampling reliable parameters. 2. The generated GNN parameters are superior to those obtained through comprehensive grid search, thoroughly exploring the potential population of "good parameters". 3. Compared to diffusion frameworks designed for general neural networks, GNN-Diff has higher quality in GNN parameter generation as it incorporates graph guidance. The paper introduces the workflow of GNN-Diff, including input graph data, parameter collection (through coarse search), training (training of parameter autoencoder, graph autoencoder, and graph conditional diffusion model), as well as parameter sampling and reconstruction. Experimental results show that the GNN models generated by GNN-Diff outperform models selected through grid search and coarse search in most cases, especially on the Cora and Actor datasets. In addition, the paper also provides visual analysis to explain why GNN-Diff can produce better prediction results.