Abstract:Recent work suggests that representations learned by adversarially robust networks are more human perceptually-aligned than non-robust networks via image manipulations. Despite appearing closer to human visual perception, it is unclear if the constraints in robust DNN representations match biological constraints found in human vision. Human vision seems to rely on texture-based/summary statistic representations in the periphery, which have been shown to explain phenomena such as crowding and performance on visual search tasks. To understand how adversarially robust optimizations/representations compare to human vision, we performed a psychophysics experiment using a set of metameric discrimination tasks where we evaluated how well human observers could distinguish between images synthesized to match adversarially robust representations compared to non-robust representations and a texture synthesis model of peripheral vision (Texforms). We found that the discriminability of robust representation and texture model images decreased to near chance performance as stimuli were presented farther in the periphery. Moreover, performance on robust and texture-model images showed similar trends within participants, while performance on non-robust representations changed minimally across the visual field. These results together suggest that (1) adversarially robust representations capture peripheral computation better than non-robust representations and (2) robust representations capture peripheral computation similar to current state-of-the-art texture peripheral vision models. More broadly, our findings support the idea that localized texture summary statistic representations may drive human invariance to adversarial perturbations and that the incorporation of such representations in DNNs could give rise to useful properties like adversarial robustness.

A Spectral View of Adversarially Robust Features

Robust spectral regression for face recognition

Feature Augmentation for Adversarial Robustness

Exploring Robust Features for Improving Adversarial Robustness

Adversarial spheres

A Fourier Perspective On Model Robustness In Computer Vision

Towards Adversarial Robustness with Multidimensional Perturbations Via Contrastive Learning

A Fourier Perspective of Feature Extraction and Adversarial Robustness

Robustness Exploration of Semantic Information in Adversarial Training

A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs

Improving Adversarial Robustness to Sensitivity and Invariance Attacks with Deep Metric Learning

Adversarial robustness of VAEs through the lens of local geometry

Provable Adversarial Robustness for Group Equivariant Tasks: Graphs, Point Clouds, Molecules, and More

Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data

Semantically Consistent Visual Representation for Adversarial Robustness

Universal Spectral Adversarial Attacks for Deformable Shapes

Robust Feature Learning for Multi-Index Models in High Dimensions

Enhancing Robust Representation in Adversarial Training: Alignment and Exclusion Criteria

Finding Biological Plausibility for Adversarially Robust Features via Metameric Tasks

Exploring the Adversarial Frontier: Quantifying Robustness via Adversarial Hypervolume

Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training