Abstract:Many real-world mission-critical applications require continual online learning from noisy data and real-time decision making with a defined confidence level. Probabilistic models and stochastic neural networks can explicitly handle uncertainty in data and allow adaptive learning-on-the-fly, but their implementation in a low-power substrate remains a challenge. Here, we introduce a novel hardware fabric that implements a new class of stochastic NN called Neural-Sampling-Machine that exploits stochasticity in synaptic connections for approximate Bayesian inference. Harnessing the inherent non-linearities and stochasticity occurring at the atomic level in emerging materials and devices allows us to capture the synaptic stochasticity occurring at the molecular level in biological synapses. We experimentally demonstrate in-silico hybrid stochastic synapse by pairing a ferroelectric field-effect transistor -based analog weight cell with a two-terminal stochastic selector element. Such a stochastic synapse can be integrated within the well-established crossbar array architecture for compute-in-memory. We experimentally show that the inherent stochastic switching of the selector element between the insulator and metallic state introduces a multiplicative stochastic noise within the synapses of NSM that samples the conductance states of the FeFET, both during learning and inference. We perform network-level simulations to highlight the salient automatic weight normalization feature introduced by the stochastic synapses of the NSM that paves the way for continual online learning without any offline Batch Normalization. We also showcase the Bayesian inferencing capability introduced by the stochastic synapse during inference mode, thus accounting for uncertainty in data. We report 98.25%accuracy on standard image classification task as well as estimation of data uncertainty in rotated samples.

Emerging Artificial Neuron Devices for Probabilistic Computing

Mott memristor based stochastic neurons for probabilistic computing

Artificial Neuronal Devices Based on Emerging Materials: Neuronal Dynamics and Applications

Emerging Devices for Biologically Accurate Neuron

A Comprehensive Review on Emerging Artificial Neuromorphic Devices

Biological plausibility and stochasticity in scalable VO2 active memristor neurons

Spiking Neurons with Neural Dynamics Implemented Using Stochastic Memristors

Toward stochastic neural computing

A Review of Artificial Spiking Neuron Devices for Neural Processing and Sensing

Stochastic Artificial Synapses Based on Nanoscale Magnetic Tunnel Junction for Neuromorphic Applications

Emerging memristive neurons for neuromorphic computing and sensing

Emerging memristive artificial neuron and synapse devices for the neuromorphic electronics era

Ultrafast Probabilistic Neuron in an Artificial Spin Ice for Robust Deep Neural Networks

Neural Sampling Machine with Stochastic Synapse allows Brain-like Learning and Inference

Progress and Benchmark of Spiking Neuron Devices and Circuits

Neuromorphic Silicon Neuron Circuits

Hardware Implementation of a Fully Functional Stochastic p‐STT Neuron for Probabilistic Computing

Emerging Optoelectronic Devices for Brain‐Inspired Computing

Artificial Synapses Enabled Neuromorphic Computing: from Blueprints to Reality

A new computing rule for neuromorphic engineering

Stochastic Computation Based Hybrid Artificial Neural Networks and Spike Neural Network