Abstract:Since the pioneering work on the lottery ticket hypothesis for graph neural networks (GNNs) was proposed in Chen et al. (2021), the study on finding graph lottery tickets (GLT) has become one of the pivotal focus in the GNN community, inspiring researchers to discover sparser GLT while achieving comparable performance to original dense networks. In parallel, the graph structure has gained substantial attention as a crucial factor in GNN training dynamics, also elucidated by several recent studies. Despite this, contemporary studies on GLT, in general, have not fully exploited inherent pathways in the graph structure and identified tickets in an iterative manner, which is time-consuming and inefficient. To address these limitations, we introduce TEDDY, a one-shot edge sparsification framework that leverages structural information by incorporating edge-degree information. Following edge sparsification, we encourage the parameter sparsity during training via simple projected gradient descent on the $\ell_0$ ball. Given the target sparsity levels for both the graph structure and the model parameters, our TEDDY facilitates efficient and rapid realization of GLT within a single training. Remarkably, our experimental results demonstrate that TEDDY significantly surpasses conventional iterative approaches in generalization, even when conducting one-shot sparsification that solely utilizes graph structures, without taking feature information into account.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to efficiently find Graph Lottery Tickets (GLT) in Graph Neural Networks (GNNs), that is, to achieve the sparsification of graph structures and model parameters through a one - time edge - sparsification method while maintaining performance. Specifically, existing research usually adopts iterative methods when looking for GLT. These methods are not only time - consuming but also inefficient and fail to fully utilize the inherent paths in the graph structure. In addition, most current research ignores the importance of graph - structure information and relies more on node features. To solve these problems, this paper proposes TEDDY, a one - time edge - sparsification framework based on a degree - discrimination strategy, aiming to use the structural information of the graph, especially the importance of low - degree edges, to maintain the main information - flow paths and encourage parameter sparsification on the ℓ0 ball through simple Projected Gradient Descent (PGD), thereby achieving the sparsification of graph structures and model parameters simultaneously in one training process. ### Main Contributions: 1. **Introducing TEDDY**: A new edge - sparsification method that maintains the integrity of the main information flow by using the structural information of the graph (especially the importance of low - degree edges). 2. **Parameter Sparsification**: Encourage parameter sparsification through Projected Gradient Descent (PGD) on the ℓ0 ball. Compared with traditional iterative methods, TEDDY only requires one training process and has lower computational costs. 3. **Experimental Verification**: Extensive experimental results show that TEDDY significantly outperforms iterative GLT methods on multiple benchmark datasets and architectures. Even when performing one - time edge - pruning without considering node features, its performance far exceeds that of baseline methods. ### Problems Solved: - **Improving Efficiency**: Through the one - time sparsification method, the high computational costs of traditional iterative methods are avoided. - **Utilizing Graph Structure**: Emphasizes the importance of graph - structure information (especially low - degree edges) in GNN training, which is often overlooked in existing research. - **Enhancing Performance**: On multiple datasets and architectures, TEDDY not only maintains the performance of the original dense network but even improves it in some cases. Through these contributions, TEDDY provides a new and effective method for GNN compression and optimization, especially when dealing with large - scale graph data, which can significantly improve computational efficiency and model performance.

TEDDY: Trimming Edges with Degree-based Discrimination strategY

Rethinking Graph Lottery Tickets: Graph Sparsity Matters

Adversarial Erasing with Pruned Elements: Towards Better Graph Lottery Ticket

Pre-Training Identification of Graph Winning Tickets in Adaptive Spatial-Temporal Graph Neural Networks

Sparse but Strong: Crafting Adversarially Robust Graph Lottery Tickets

Fast Track to Winning Tickets: Repowering One-Shot Pruning for Graph Neural Networks

Early-Bird GCNs: Graph-Network Co-Optimization Towards More Efficient GCN Training and Inference via Drawing Early-Bird Lottery Tickets

Graph Structure Learning Via Lottery Hypothesis at Scale.

Multicoated and Folded Graph Neural Networks with Strong Lottery Tickets

Two Heads Are Better Than One: Boosting Graph Sparse Training via Semantic and Topological Awareness

Exploring Lottery Ticket Hypothesis in Spiking Neural Networks

Coarsening the Granularity: Towards Structurally Sparse Lottery Tickets

Efficient Topology-aware Data Augmentation for High-Degree Graph Neural Networks

Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs

Pursing the Sparse Limitation of Spiking Deep Learning Structures

Interpretable Sparsification of Brain Graphs: Better Practices and Effective Designs for Graph Neural Networks

ADEdgeDrop: Adversarial Edge Dropping for Robust Graph Neural Networks

EiG-Search: Generating Edge-Induced Subgraphs for GNN Explanation in Linear Time

Spectral Graph Pruning Against Over-Squashing and Over-Smoothing

Training Robust Graph Neural Networks with Topology Adaptive Edge Dropping

Edge Convolutional Networks: Decomposing Graph Convolutional Networks for Stochastic Training with Independent Edges