Abstract:Spiking Neural Networks (SNNs) have attracted great attention for their energy-efficient operations and biologically inspired structures, offering potential advantages over Artificial Neural Networks (ANNs) in terms of energy efficiency and interpretability. Nonetheless, similar to ANNs, the robustness of SNNs remains a challenge, especially when facing adversarial attacks. Existing techniques, whether adapted from ANNs or specifically designed for SNNs, exhibit limitations in training SNNs or defending against strong attacks. In this paper, we propose a novel approach to enhance the robustness of SNNs through gradient sparsity regularization. We observe that SNNs exhibit greater resilience to random perturbations compared to adversarial perturbations, even at larger scales. Motivated by this, we aim to narrow the gap between SNNs under adversarial and random perturbations, thereby improving their overall robustness. To achieve this, we theoretically prove that this performance gap is upper bounded by the gradient sparsity of the probability associated with the true label concerning the input image, laying the groundwork for a practical strategy to train robust SNNs by regularizing the gradient sparsity. We validate the effectiveness of our approach through extensive experiments on both image-based and event-based datasets. The results demonstrate notable improvements in the robustness of SNNs. Our work highlights the importance of gradient sparsity in SNNs and its role in enhancing robustness.

Training Robust Models Using Random Projection

Robustra: Training Provable Robust Neural Networks over Reference Adversarial Space.

Enhanced Expressive Power and Fast Training of Neural Networks by Random Projections

Robust Sparse Regularization: Simultaneously Optimizing Neural Network Robustness and Compactness

Towards Robust Training of Neural Networks by Regularizing Adversarial Gradients

Random Projections For Large-Scale Regression

Random Linear Projections Loss for Hyperplane-Based Optimization in Neural Networks

Sparse Data-Driven Random Projection in Regression for High-Dimensional Data

Random projections: data perturbation for classification problems

Regularizers for Single-step Adversarial Training

Random Projection in Deep Neural Networks

Promoting Robustness of Randomized Smoothing: Two Cost-Effective Approaches

Random Projections for Classification: A Recovery Approach

Projected Randomized Smoothing for Certified Adversarial Robustness

RANDOM MASK: Towards Robust Convolutional Neural Networks

Hierarchical Feature Selection for Random Projection

Accelerated Sparse Linear Regression Via Random Projection.

Regularized Training and Tight Certification for Randomized Smoothed Classifier with Provable Robustness

Enhancing Adversarial Robustness in SNNs with Sparse Gradients

Training neural networks on high-dimensional data using random projection

A constrained optimization approach to improve robustness of neural networks