Abstract:Training high-quality deep models necessitates vast amounts of data, resulting in overwhelming computational and memory demands. Recently, data pruning, distillation, and coreset selection have been developed to streamline data volume by retaining, synthesizing, or selecting a small yet informative subset from the full set. Among these methods, data pruning incurs the least additional training cost and offers the most practical acceleration benefits. However, it is the most vulnerable, often suffering significant performance degradation with imbalanced or biased data schema, thus raising concerns about its accuracy and reliability in on-device deployment. Therefore, there is a looming need for a new data pruning paradigm that maintains the efficiency of previous practices while ensuring balance and robustness. Unlike the fields of computer vision and natural language processing, where mature solutions have been developed to address these issues, graph neural networks (GNNs) continue to struggle with increasingly large-scale, imbalanced, and noisy datasets, lacking a unified dataset pruning solution. To achieve this, we introduce a novel dynamic soft-pruning method, GDeR, designed to update the training ``basket'' during the process using trainable prototypes. GDeR first constructs a well-modeled graph embedding hypersphere and then samples \textit{representative, balanced, and unbiased subsets} from this embedding space, which achieves the goal we called Graph Training Debugging. Extensive experiments on five datasets across three GNN backbones, demonstrate that GDeR (I) achieves or surpasses the performance of the full dataset with 30%~50% fewer training samples, (II) attains up to a 2.81x lossless training speedup, and (III) outperforms state-of-the-art pruning methods in imbalanced training and noisy training scenarios by 0.3%~4.3% and 3.6%~7.8%, respectively.

Training Sparse Graph Neural Networks Via Pruning and Sprouting

Two Heads Are Better Than One: Boosting Graph Sparse Training via Semantic and Topological Awareness

Pruning graph neural networks by evaluating edge properties

Graph Sparsification via Mixture of Graphs

GDeR: Safeguarding Efficiency, Balancing, and Robustness via Prototypical Graph Pruning

RGP: Neural Network Pruning Through Regular Graph With Edges Swapping

Sparse optimization guided pruning for neural networks

Sparse Training via Boosting Pruning Plasticity with Neuroregeneration

Interpretable Sparsification of Brain Graphs: Better Practices and Effective Designs for Graph Neural Networks

Improving the interpretability of GNN predictions through conformal-based graph sparsification

Towards Lightweight Graph Neural Network Search with Curriculum Graph Sparsification

Graph Sparsification for Enhanced Conformal Prediction in Graph Neural Networks

Rethinking Graph Lottery Tickets: Graph Sparsity Matters

RGP: Neural Network Pruning through Its Regular Graph Structure

Reliable and Compact Graph Fine-tuning via GraphSparse Prompting

Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks

Robust Graph Representation Learning Via Neural Sparsification.

An End-to-End Network Pruning Pipeline with Sparsity Enforcement

Sparse Decomposition of Graph Neural Networks

One-Cycle Pruning: Pruning ConvNets Under a Tight Training Budget

Fast Track to Winning Tickets: Repowering One-Shot Pruning for Graph Neural Networks