Abstract:Randomized smoothing is a defensive technique to achieve enhanced robustness against adversarial examples which are small input perturbations that degrade the performance of neural network models. Conventional randomized smoothing adds random noise with a fixed noise level for every input sample to smooth out adversarial perturbations. This paper proposes a new variational framework that uses a per-sample noise level suitable for each input by introducing a noise level selector. Our experimental results demonstrate enhancement of empirical robustness against adversarial attacks. We also provide and analyze the certified robustness for our sample-wise smoothing method.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to improve the robustness of neural network models against adversarial sample attacks. Specifically, it aims to improve the Randomized Smoothing technique to enhance the adversarial robustness of each input sample. The traditional randomized smoothing method smoothes all input samples with a fixed noise level, while this paper proposes a new variational randomized smoothing framework. By introducing a noise - level selector to select an appropriate noise level for each input sample, it can better deal with adversarial sample attacks. ### Main Problems and Solutions 1. **Vulnerability to Adversarial Sample Attacks**: - Neural networks are vulnerable to adversarial sample attacks, which reduce the model performance by adding small perturbations to the input data. - For example, the Projected Gradient Descent (PGD) attack is a common white - box attack method that uses the weight and gradient information of the target neural network to generate adversarial samples. 2. **Limitations of Traditional Randomized Smoothing**: - The traditional randomized smoothing method smoothes all input samples with the same noise level, which may lead to insufficient robustness in some cases. 3. **Proposed Solutions**: - **Variational Randomized Smoothing Framework**: Introduce a noise - level selector to select an appropriate noise level according to the characteristics of each input sample. - **General Training Scheme**: Use a random regularization method to train the selector so that it can adapt to different conditions and produce different noise intensities. - **Conditional Meta - Learning**: Allow users to freely adjust the noise intensity by specifying the parameter λ at test time without retraining. - **Dual - Smoothing Defense**: To prevent the selector itself from becoming an attack target, a dual - smoothing method is proposed to protect the selector and the base classifier. - **Modified Certified Robustness**: Provide improved certified robustness based on the boundaries of median smoothing. ### Formula Explanation - **Idealized Smoothing Classifier**: \[ g(x):=\arg\max_{c\in C}P[f(x + \varepsilon)=c] \] where \(\varepsilon\sim N(0,\sigma_s^2I_d)\), and \(I_d\) is the \(d\)-dimensional identity matrix. - **Certified Robustness**: \[ R = \frac{\sigma_s}{2}(\Phi^{-1}(p_a)-\Phi^{-1}(p_b)) \] where \(\Phi^{-1}\) is the inverse function of the standard Gaussian cumulative distribution function (CDF), and \(p_a\) and \(p_b\) are the probabilities of the two most likely outputs respectively. - **KL - Divergence Regularization**: \[ D_{\text{KL}}(p\parallel q)=d\left[\frac{1}{2}\left(\frac{\sigma_s}{\sigma_t}\right)^2-\frac{1}{2}-\log\left(\frac{\sigma_s}{\sigma_t}\right)\right] \] - **Loss Function**: \[ L=(1 - \lambda)L_{\text{CE}}(x,y)+\lambda D_{\text{KL}}(p\parallel q) \] Through these improvements, the paper shows the improvement of its method in empirical robustness and certified robustness, and verifies its effectiveness in different attack scenarios.

Variational Randomized Smoothing for Sample-Wise Adversarial Robustness

Treatment of Statistical Estimation Problems in Randomized Smoothing for Adversarial Robustness

Smoothed Inference for Adversarially-Trained Models

Randomized Smoothing Under Attack: How Good is it in Practice?

Projected Randomized Smoothing for Certified Adversarial Robustness

Certified Adversarial Robustness of Machine Learning-based Malware Detectors via (De)Randomized Smoothing

Adaptive Randomized Smoothing: Certified Adversarial Robustness for Multi-Step Defences

Estimating the Robustness Radius for Randomized Smoothing with 100$\times$ Sample Efficiency

Promoting Robustness of Randomized Smoothing: Two Cost-Effective Approaches

Analyzing Accuracy Loss in Randomized Smoothing Defenses

Certified Adversarial Robustness Under the Bounded Support Set.

Advancing the Robustness of Large Language Models through Self-Denoised Smoothing

Understanding the Robustness of Randomized Feature Defense Against Query-Based Adversarial Attacks

Certified Adversarial Robustness via Partition-based Randomized Smoothing

Dynamic Defense Approach for Adversarial Robustness in Deep Neural Networks via Stochastic Ensemble Smoothed Model

Sequential Randomized Smoothing for Adversarially Robust Speech Recognition

Rethinking Randomized Smoothing from the Perspective of Scalability

Tight Certificates of Adversarial Robustness for Randomly Smoothed Classifiers

Towards Bridging the gap between Empirical and Certified Robustness against Adversarial Examples

Adversarial sample attack method based on loss smoothing

Certified Adversarial Robustness Within Multiple Perturbation Bounds