Diffusing to the Top: Boost Graph Neural Networks with Minimal Hyperparameter Tuning

Lequan Lin,Dai Shi,Andi Han,Zhiyong Wang,Junbin Gao
2024-10-08
Abstract:Graph Neural Networks (GNNs) are proficient in graph representation learning and achieve promising performance on versatile tasks such as node classification and link prediction. Usually, a comprehensive hyperparameter tuning is essential for fully unlocking GNN's top performance, especially for complicated tasks such as node classification on large graphs and long-range graphs. This is usually associated with high computational and time costs and careful design of appropriate search spaces. This work introduces a graph-conditioned latent diffusion framework (GNN-Diff) to generate high-performing GNNs based on the model checkpoints of sub-optimal hyperparameters selected by a light-tuning coarse search. We validate our method through 166 experiments across four graph tasks: node classification on small, large, and long-range graphs, as well as link prediction. Our experiments involve 10 classic and state-of-the-art target models and 20 publicly available datasets. The results consistently demonstrate that GNN-Diff: (1) boosts the performance of GNNs with efficient hyperparameter tuning; and (2) presents high stability and generalizability on unseen data across multiple generation runs. The code is available at <a class="link-external link-https" href="https://github.com/lequanlin/GNN-Diff" rel="external noopener nofollow">this https URL</a>.
Machine Learning
What problem does this paper attempt to address?
The problem that this paper attempts to solve is that graph neural networks (GNNs) require a large amount of hyper - parameter tuning when dealing with complex tasks. Although GNNs perform well in simple tasks, in order to achieve the best performance in node classification on complex tasks such as large - scale graphs and long - range graphs, extensive hyper - parameter tuning is usually required. This tuning not only consumes a large amount of computing resources and time, but also requires careful design of the search space. Therefore, the author proposes a graph - conditional latent diffusion framework (GNN - Diff), aiming to improve the performance of GNNs by minimizing hyper - parameter tuning. Specifically, GNN - Diff achieves this goal through the following steps: 1. **Parameter collection**: Use a coarse search to select an appropriate hyper - parameter configuration and collect the model checkpoints generated during the training process. 2. **Training**: First, train the parameter auto - encoder (PAE) and the graph auto - encoder (GAE), and then train the graph - conditional latent diffusion model (G - LDM) to recover the latent parameters from white noise. 3. **Inference and prediction**: Sample latent parameters from G - LDM, use the PAE decoder to reconstruct the GNN parameters in the original parameter space, and return them to the target GNN for prediction. Through this method, GNN - Diff can significantly improve the performance of GNNs while reducing the cost of hyper - parameter tuning, especially in tasks dealing with large - scale and long - range graphs. Experimental results show that GNN - Diff can not only efficiently improve the performance of GNNs, but also exhibits high stability and generalization ability.