Abstract:To improve the robustness of deep classifiers against adversarial perturbations, many approaches have been proposed, such as designing new architectures with better robustness properties (e.g., Lipschitz-capped networks), or modifying the training process itself (e.g., min-max optimization, constrained learning, or regularization). These approaches, however, might not be effective at increasing the margin in the input (feature) space. As a result, there has been an increasing interest in developing training procedures that can directly manipulate the decision boundary in the input space. In this paper, we build upon recent developments in this category by developing a robust training algorithm whose objective is to increase the margin in the output (logit) space while regularizing the Lipschitz constant of the model along vulnerable directions. We show that these two objectives can directly promote larger margins in the input space. To this end, we develop a scalable method for calculating guaranteed differentiable upper bounds on the Lipschitz constant of neural networks accurately and efficiently. The relative accuracy of the bounds prevents excessive regularization and allows for more direct manipulation of the decision boundary. Furthermore, our Lipschitz bounding algorithm exploits the monotonicity and Lipschitz continuity of the activation layers, and the resulting bounds can be used to design new layers with controllable bounds on their Lipschitz constant. Experiments on the MNIST, CIFAR-10, and Tiny-ImageNet data sets verify that our proposed algorithm obtains competitively improved results compared to the state-of-the-art.

Towards Certifying L Robustness Using Neural Networks with L-Dist Neurons

Towards Certifying $\ell_\infty$ Robustness Using Neural Networks with $\Ell_\infty$-Dist Neurons

Towards certifying ℓ∞ robustness using neural networks with ℓ∞-dist neurons

BOOSTING THE CERTIFIED ROBUSTNESS OF L-INFINITY DISTANCE NETS

Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspective

Globally-Robust Neural Networks

A Recipe for Improved Certifiable Robustness

Towards Certifying the Asymmetric Robustness for Neural Networks: Quantification and Applications

Robustra: Training Provable Robust Neural Networks over Reference Adversarial Space.

Towards the first adversarially robust neural network model on MNIST

SoK: Certified Robustness for Deep Neural Networks

Towards Certified Probabilistic Robustness with High Accuracy

Certified Robustness via Dynamic Margin Maximization and Improved Lipschitz Regularization

Towards Evaluating the Robustness of Neural Networks

Certifying Global Robustness for Deep Neural Networks

Certifying Robustness of Convolutional Neural Networks with Tight Linear Approximation

Tight Certified Robustness via Min-Max Representations of ReLU Neural Networks

Adversarial Robustness Certification for Bayesian Neural Networks

Input Validation for Neural Networks via Runtime Local Robustness Verification

Certified Robustness to Programmable Transformations in LSTMs