Abstract:Many real-world mission-critical applications require continual online learning from noisy data and real-time decision making with a defined confidence level. Probabilistic models and stochastic neural networks can explicitly handle uncertainty in data and allow adaptive learning-on-the-fly, but their implementation in a low-power substrate remains a challenge. Here, we introduce a novel hardware fabric that implements a new class of stochastic NN called Neural-Sampling-Machine that exploits stochasticity in synaptic connections for approximate Bayesian inference. Harnessing the inherent non-linearities and stochasticity occurring at the atomic level in emerging materials and devices allows us to capture the synaptic stochasticity occurring at the molecular level in biological synapses. We experimentally demonstrate in-silico hybrid stochastic synapse by pairing a ferroelectric field-effect transistor -based analog weight cell with a two-terminal stochastic selector element. Such a stochastic synapse can be integrated within the well-established crossbar array architecture for compute-in-memory. We experimentally show that the inherent stochastic switching of the selector element between the insulator and metallic state introduces a multiplicative stochastic noise within the synapses of NSM that samples the conductance states of the FeFET, both during learning and inference. We perform network-level simulations to highlight the salient automatic weight normalization feature introduced by the stochastic synapses of the NSM that paves the way for continual online learning without any offline Batch Normalization. We also showcase the Bayesian inferencing capability introduced by the stochastic synapse during inference mode, thus accounting for uncertainty in data. We report 98.25%accuracy on standard image classification task as well as estimation of data uncertainty in rotated samples.

Two-dimensional materials-based probabilistic synapses and reconfigurable neurons for measuring inference uncertainty using Bayesian neural networks

Bayesian Inference Based Robust Computing on Memristor Crossbar

BRoCoM: A Bayesian Framework for Robust Computing on Memristor Crossbar.

Gaussian synapses for probabilistic neural networks

Neural Sampling Machine with Stochastic Synapse allows Brain-like Learning and Inference

A Design Methodology for Fault-Tolerant Neuromorphic Computing Using Bayesian Neural Network

Spiking neural networks based on two-dimensional materials

Unsupervised learning in hexagonal boron nitride memristor-based spiking neural networks

Emerging Artificial Neuron Devices for Probabilistic Computing

Reconfigurable Stochastic Neurons Based on Strain Engineered Low Barrier Nanomagnets

Bayes2IMC: In-Memory Computing for Bayesian Binary Neural Networks

Mott memristor based stochastic neurons for probabilistic computing

Few-Shot Testing: Estimating Uncertainty of Memristive Deep Neural Networks Using One Bayesian Test Vector

Exploiting Inherent Error-Resiliency of Neuromorphic Computing to achieve Extreme Energy-Efficiency through Mixed-Signal Neurons

Self-similar Magneto-electric Nanocircuit Technology for Probabilistic Inference Engines

Uncertainty quantification via a memristor Bayesian deep neural network for risk-sensitive reinforcement learning

Enhancing Reliability of Neural Networks at the Edge: Inverted Normalization with Stochastic Affine Transformations

HXNOR-PBNN: A Scalable and Parallel Spintronics Synaptic Architecture for Probabilistic Binary Neural Networks

An All-in-One Bioinspired Neural Network

Improving classification accuracy of feedforward neural networks for spiking neuromorphic chips