Abstract:Analog physical neural networks, which hold promise for improved energy efficiency and speed compared to digital electronic neural networks, are nevertheless typically operated in a relatively high-power regime so that the signal-to-noise ratio (SNR) is large (>10). What happens if an analog system is instead operated in an ultra-low-power regime, in which the behavior of the system becomes highly stochastic and the noise is no longer a small perturbation on the signal? In this paper, we study this question in the setting of optical neural networks operated in the limit where some layers use only a single photon to cause a neuron activation. Neuron activations in this limit are dominated by quantum noise from the fundamentally probabilistic nature of single-photon detection of weak optical signals. We show that it is possible to train stochastic optical neural networks to perform deterministic image-classification tasks with high accuracy in spite of the extremely high noise (SNR ~ 1) by using a training procedure that directly models the stochastic behavior of photodetection. We experimentally demonstrated MNIST classification with a test accuracy of 98% using an optical neural network with a hidden layer operating in the single-photon regime; the optical energy used to perform the classification corresponds to 0.008 photons per multiply-accumulate (MAC) operation, which is equivalent to 0.003 attojoules of optical energy per MAC. Our experiment used >40x fewer photons per inference than previous state-of-the-art low-optical-energy demonstrations, to achieve the same accuracy of >90%. Our work shows that some extremely stochastic analog systems, including those operating in the limit where quantum noise dominates, can nevertheless be used as layers in neural networks that deterministically perform classification tasks with high accuracy if they are appropriately trained.

Stochastic resonance neurons in artificial neural networks

STOCHASTIC RESONANCE IN A DOUBLE LAYER HODGKIN-HUXLEY NEURONAL NETWORK

Effect of noises on two-layer hodgkin-huxley neuronal network

Stochastic Resonance Enhancing Detectability Of Weak Signal By Neuronal Networks Model For Receiver

Stochastic cellular automata model of neural networks

Stochastic Resonance In A Globally Coupled Neuronal Network

Stochastic Resonance in a Coupled Neuronal Network

Stochastic and Coherence Resonance in Feed-Forward-Loop Neuronal Network Motifs

Noise-Resilient Designs for Optical Neural Networks

Control of mediated stochastic resonance in multilayer neural networks

Stochastic resonance in Hodgkin-Huxley neuron induced by unreliable synaptic transmission.

Stochastic Resonance in Feedforward-Loop Neuronal Network Motifs in Astrocyte Field

Exploiting Noise As a Resource for Computation and Learning in Spiking Neural Networks

Inverse stochastic resonance in adaptive small-world neural networks

Robust Stability and Robust Periodicity of Delayed Recurrent Neural Networks with Noise Disturbance

Stochastic Resonance Tuned By Correlations In Neural Models

The silencing of neuronal activity by noise and the phenomenon of inverse stochastic resonance

Learning biological neuronal networks with artificial neural networks: neural oscillations

Quantum-noise-limited optical neural networks operating at a few quanta per activation

Learning spiking neuronal networks with artificial neural networks: neural oscillations

Understanding Activation Patterns in Artificial Neural Networks by Exploring Stochastic Processes