Abstract:Training high-quality deep models necessitates vast amounts of data, resulting in overwhelming computational and memory demands. Recently, data pruning, distillation, and coreset selection have been developed to streamline data volume by retaining, synthesizing, or selecting a small yet informative subset from the full set. Among these methods, data pruning incurs the least additional training cost and offers the most practical acceleration benefits. However, it is the most vulnerable, often suffering significant performance degradation with imbalanced or biased data schema, thus raising concerns about its accuracy and reliability in on-device deployment. Therefore, there is a looming need for a new data pruning paradigm that maintains the efficiency of previous practices while ensuring balance and robustness. Unlike the fields of computer vision and natural language processing, where mature solutions have been developed to address these issues, graph neural networks (GNNs) continue to struggle with increasingly large-scale, imbalanced, and noisy datasets, lacking a unified dataset pruning solution. To achieve this, we introduce a novel dynamic soft-pruning method, GDeR, designed to update the training ``basket'' during the process using trainable prototypes. GDeR first constructs a well-modeled graph embedding hypersphere and then samples \textit{representative, balanced, and unbiased subsets} from this embedding space, which achieves the goal we called Graph Training Debugging. Extensive experiments on five datasets across three GNN backbones, demonstrate that GDeR (I) achieves or surpasses the performance of the full dataset with 30%~50% fewer training samples, (II) attains up to a 2.81x lossless training speedup, and (III) outperforms state-of-the-art pruning methods in imbalanced training and noisy training scenarios by 0.3%~4.3% and 3.6%~7.8%, respectively.

Jump-GRS: a multi-phase approach to structured pruning of neural networks for neural decoding

Efficient Structure Slimming for Spiking Neural Networks

Class-Aware Pruning for Efficient Neural Networks

Pruning of Deep Spiking Neural Networks Through Gradient Rewiring.

Structured Pruning of Recurrent Neural Networks through Neuron Selection

Continuous Neural Decoding Method Based on General Regression Neural Network.

A Sparsity-Adapted Hardware Implementation of SNN for Cortical Spike Trains Decoding

RGP: Neural Network Pruning Through Regular Graph With Edges Swapping

Neural Network Pruning by Gradient Descent

Temporal Spiking Generative Adversarial Networks for Heading Direction Decoding

Towards Efficient Deep Spiking Neural Networks Construction with Spiking Activity based Pruning

GenExp: Multi-objective pruning for deep neural network based on genetic algorithm

Scalable iterative pruning of large language and vision models using block coordinate descent

Structurally Prune Anything: Any Architecture, Any Framework, Any Time

GDeR: Safeguarding Efficiency, Balancing, and Robustness via Prototypical Graph Pruning

Efficient DNN Neuron Pruning by Minimizing Layer-wise Nonlinear Reconstruction Error

Sparse optimization guided pruning for neural networks

An efficient pruning and fine-tuning method for deep spiking neural network

Evolving Transferable Neural Pruning Functions

FGGP: Fixed-Rate Gradient-First Gradual Pruning

Brain-Inspired Efficient Pruning: Exploiting Criticality in Spiking Neural Networks