Abstract:To improve the robustness of deep classifiers against adversarial perturbations, many approaches have been proposed, such as designing new architectures with better robustness properties (e.g., Lipschitz-capped networks), or modifying the training process itself (e.g., min-max optimization, constrained learning, or regularization). These approaches, however, might not be effective at increasing the margin in the input (feature) space. As a result, there has been an increasing interest in developing training procedures that can directly manipulate the decision boundary in the input space. In this paper, we build upon recent developments in this category by developing a robust training algorithm whose objective is to increase the margin in the output (logit) space while regularizing the Lipschitz constant of the model along vulnerable directions. We show that these two objectives can directly promote larger margins in the input space. To this end, we develop a scalable method for calculating guaranteed differentiable upper bounds on the Lipschitz constant of neural networks accurately and efficiently. The relative accuracy of the bounds prevents excessive regularization and allows for more direct manipulation of the decision boundary. Furthermore, our Lipschitz bounding algorithm exploits the monotonicity and Lipschitz continuity of the activation layers, and the resulting bounds can be used to design new layers with controllable bounds on their Lipschitz constant. Experiments on the MNIST, CIFAR-10, and Tiny-ImageNet data sets verify that our proposed algorithm obtains competitively improved results compared to the state-of-the-art.

BOOSTING THE CERTIFIED ROBUSTNESS OF L-INFINITY DISTANCE NETS

Boosting the Certified Robustness of L-infinity Distance Nets

Towards Certifying L Robustness Using Neural Networks with L-Dist Neurons

Towards Certifying $\ell_\infty$ Robustness Using Neural Networks with $\Ell_\infty$-Dist Neurons

Towards certifying ℓ∞ robustness using neural networks with ℓ∞-dist neurons

Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspective

A Recipe for Improved Certifiable Robustness

Certifying Out-of-Domain Generalization for Blackbox Functions

Certified Robustness via Dynamic Margin Maximization and Improved Lipschitz Regularization

Improved techniques for deterministic l2 robustness

Adaptive Certified Training: Towards Better Accuracy-Robustness Tradeoffs

Certifying Robustness of Convolutional Neural Networks with Tight Linear Approximation

Boosting Verified Training for Robust Image Classifications via Abstraction

Understanding Certified Training with Interval Bound Propagation

Certified Adversarial Robustness Under the Bounded Support Set.

Unlocking Deterministic Robustness Certification on ImageNet

Tightened Convex Relaxations for Neural Network Robustness Certification

Globally-Robust Neural Networks

Computable Lipschitz Bounds for Deep Neural Networks

Asymmetric Certified Robustness via Feature-Convex Neural Networks

Improving the Tightness of Convex Relaxation Bounds for Training Certifiably Robust Classifiers