Abstract:Deep neural networks (DNN) trained in a supervised way suffer from two known problems. First, the minima of the objective function used in learning correspond to data points (also known as rubbish examples or fooling images) that lack semantic similarity with the training data. Second, a clean input can be changed by a small, and often imperceptible for human vision, perturbation, so that the resulting deformed input is misclassified by the network. These findings emphasize the differences between the ways DNN and humans classify patterns, and raise a question of designing learning algorithms that more accurately mimic human perception compared to the existing methods. Our paper examines these questions within the framework of Dense Associative Memory (DAM) models. These models are defined by the energy function, with higher order (higher than quadratic) interactions between the neurons. We show that in the limit when the power of the interaction vertex in the energy function is sufficiently large, these models have the following three properties. First, the minima of the objective function are free from rubbish images, so that each minimum is a semantically meaningful pattern. Second, artificial patterns poised precisely at the decision boundary look ambiguous to human subjects and share aspects of both classes that are separated by that decision boundary. Third, adversarial images constructed by models with small power of the interaction vertex, which are equivalent to DNN with rectified linear units (ReLU), fail to transfer to and fool the models with higher order interactions. This opens up a possibility to use higher order models for detecting and stopping malicious adversarial attacks. The presented results suggest that DAM with higher order energy functions are closer to human visual perception than DNN with ReLUs.

The Inherent Adversarial Robustness of Analog In-Memory Computing

Attack As Defense: Characterizing Adversarial Examples Using Robustness.

Mitigating RC-Delay Induced Accuracy Loss in Analog In-Memory Computing: A Non-Compromising Approach

AnalogNAS: A Neural Network Design Framework for Accurate Inference with Analog In-Memory Computing

Accuracy and Resiliency of Analog Compute-in-Memory Inference Engines

Mitigating Adversarial Attack for Compute-in-Memory Accelerator Utilizing On-chip Finetune

AD2VNCS: Adversarial Defense and Device Variation-tolerance in Memristive Crossbar-based Neuromorphic Computing Systems

A 64-core mixed-signal in-memory compute chip based on phase-change memory for deep neural network inference

AD 2 VNCS: A Dversarial D Efense and D Evice V Ariation-Tolerance in Memristive Crossbar-based N Euromorphic C Omputing S Ystems

Nonideality-aware training makes memristive networks more robust to adversarial attacks

The Unlikely Hero: Nonideality in Analog Photonic Neural Networks as Built-in Defender Against Adversarial Attacks

Hardware Accelerator for Adversarial Attacks on Deep Learning Neural Networks

PowerGAN: A Machine Learning Approach for Power Side-Channel Attack on Compute-in-Memory Accelerators

Noisy Computations during Inference: Harmful or Helpful?

Dense Associative Memory is Robust to Adversarial Inputs

Towards Robustifying Image Classifiers against the Perils of Adversarial Attacks on Artificial Intelligence Systems

Using Undervolting as an On-Device Defense Against Adversarial Machine Learning Attacks

When Side-Channel Attacks Break the Black-Box Property of Embedded Artificial Intelligence

DNN-Defender: An in-DRAM Deep Neural Network Defense Mechanism for Adversarial Weight Attack

HASI: Hardware-Accelerated Stochastic Inference, A Defense Against Adversarial Machine Learning Attacks

Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators