Overcoming the noise in neural computing

James B Aimone,Sapan Agarwal,James B. Aimone

DOI: https://doi.org/10.1126/science.adn8545

IF: 56.9

2024-02-23

Science

Abstract:Circuit strategies can enable noisy analog hardware to achieve high precision

multidisciplinary sciences

What problem does this paper attempt to address?

Noisy Machines: Understanding Noisy Neural Networks and Enhancing Robustness to Analog Hardware Errors Using Distillation

Chuteng Zhou,Prad Kadambi,Matthew Mattina,Paul N. Whatmough

DOI: https://doi.org/10.48550/arXiv.2001.04974

IF: 5.414

2020-01-14

Machine Learning

Abstract:The success of deep learning has brought forth a wave of interest in computer hardware design to better meet the high demands of neural network inference. In particular, analog computing hardware has been heavily motivated specifically for accelerating neural networks, based on either electronic, optical or photonic devices, which may well achieve lower power consumption than conventional digital electronics. However, these proposed analog accelerators suffer from the intrinsic noise generated by their physical components, which makes it challenging to achieve high accuracy on deep neural networks. Hence, for successful deployment on analog accelerators, it is essential to be able to train deep neural networks to be robust to random continuous noise in the network weights, which is a somewhat new challenge in machine learning. In this paper, we advance the understanding of noisy neural networks. We outline how a noisy neural network has reduced learning capacity as a result of loss of mutual information between its input and output. To combat this, we propose using knowledge distillation combined with noise injection during training to achieve more noise robust networks, which is demonstrated experimentally across different networks and datasets, including ImageNet. Our method achieves models with as much as two times greater noise tolerance compared with the previous best attempts, which is a significant step towards making analog hardware practical for deep learning.
Exploiting Noise As a Resource for Computation and Learning in Spiking Neural Networks

Gehua Ma,Rui Yan,Huajin Tang

DOI: https://doi.org/10.1016/j.patter.2023.100831

2023-01-01

Abstract:Networks of spiking neurons underpin the extraordinary information-processing capabilities of the brain and have become pillar models in neuromorphic artificial intelligence. Despite extensive research on spiking neural networks (SNNs), most studies are established on deterministic models, overlooking the inherent non-deterministic, noisy nature of neural computations. This study introduces the noisy SNN (NSNN) and the noise-driven learning (NDL) rule by incorporating noisy neuronal dynamics to exploit the computational advantages of noisy neural processing. The NSNN provides a theoretical framework that yields scalable, flexible, and reliable computation and learning. We demonstrate that this framework leads to spiking neural models with competitive performance, improved robustness against challenging perturbations compared with deterministic SNNs, and better reproducing probabilistic computation in neural coding. Generally, this study offers a powerful and easy-to-use tool for machine learning, neuromorphic intelligence practitioners, and computational neuroscience researchers.
Understanding and mitigating noise in trained deep neural networks

Nadezhda Semenova,Laurent Larger,Daniel Brunner

DOI: https://doi.org/10.1016/j.neunet.2021.11.008

2021-12-16

Abstract:Deep neural networks unlocked a vast range of new applications by solving tasks of which many were previously deemed as reserved to higher human intelligence. One of the developments enabling this success was a boost in computing power provided by special purpose hardware, such as graphic or tensor processing units. However, these do not leverage fundamental features of neural networks like parallelism and analog state variables. Instead, they emulate neural networks relying on binary computing, which results in unsustainable energy consumption and comparatively low speed. Fully parallel and analogue hardware promises to overcome these challenges, yet the impact of analogue neuron noise and its propagation, i.e. accumulation, threatens rendering such approaches inept. Here, we determine for the first time the propagation of noise in deep neural networks comprising noisy nonlinear neurons in trained fully connected layers. We study additive and multiplicative as well as correlated and uncorrelated noise, and develop analytical methods that predict the noise level in any layer of symmetric deep neural networks or deep neural networks trained with back propagation. We find that noise accumulation is generally bound, and adding additional network layers does not worsen the signal to noise ratio beyond a limit. Most importantly, noise accumulation can be suppressed entirely when neuron activation functions have a slope smaller than unity. We therefore developed the framework for noise in fully connected deep neural networks implemented in analog systems, and identify criteria allowing engineers to design noise-resilient novel neural network hardware.

Neural and Evolutionary Computing
Noisy Computations during Inference: Harmful or Helpful?

Minghai Qin,Dejan Vucinic

DOI: https://doi.org/10.48550/arXiv.1811.10649

2018-11-27

Abstract:We study two aspects of noisy computations during inference. The first aspect is how to mitigate their side effects for naturally trained deep learning systems. One of the motivations for looking into this problem is to reduce the high power cost of conventional computing of neural networks through the use of analog neuromorphic circuits. Traditional GPU/CPU-centered deep learning architectures exhibit bottlenecks in power-restricted applications (e.g., embedded systems). The use of specialized neuromorphic circuits, where analog signals passed through memory-cell arrays are sensed to accomplish matrix-vector multiplications, promises large power savings and speed gains but brings with it the problems of limited precision of computations and unavoidable analog noise. We manage to improve inference accuracy from 21.1% to 99.5% for MNIST images, from 29.9% to 89.1% for CIFAR10, and from 15.5% to 89.6% for MNIST stroke sequences with the presence of strong noise (with signal-to-noise power ratio being 0 dB) by noise-injected training and a voting method. This observation promises neural networks that are insensitive to inference noise, which reduces the quality requirements on neuromorphic circuits and is crucial for their practical usage. The second aspect is how to utilize the noisy inference as a defensive architecture against black-box adversarial attacks. During inference, by injecting proper noise to signals in the neural networks, the robustness of adversarially-trained neural networks against black-box attacks has been further enhanced by 0.5% and 1.13% for two adversarially trained models for MNIST and CIFAR10, respectively.

Machine Learning,Cryptography and Security,Computer Vision and Pattern Recognition
Noise mitigation strategies in physical feedforward neural networks

Nadezhda Semenova,Daniel Brunner

DOI: https://doi.org/10.1063/5.0096637

2022-05-18

Abstract:Physical neural networks are promising candidates for next generation artificial intelligence hardware. In such architectures, neurons and connections are physically realized and do not leverage digital concepts with their practically infinite signal-to-noise ratio to encode, transduce and transform information. They therefore are prone to noise with a variety of statistical and architectural properties, and effective strategies leveraging network-inherent assets to mitigate noise in an hardware-efficient manner are important in the pursuit of next generation neural network hardware. Based on analytical derivations, we here introduce and analyse a variety of different noise-mitigation approaches. We analytically show that intra-layer connections in which the connection matrix's squared mean exceeds the mean of its square fully suppresses uncorrelated noise. We go beyond and develop two synergistic strategies for noise that is uncorrelated and correlated across populations of neurons. First, we introduce the concept of ghost neurons, where each group of neurons perturbed by correlated noise has a negative connection to a single neuron, yet without receiving any input information. Secondly, we show that pooling of neuron populations is an efficient approach to suppress uncorrelated noise. As such, we developed a general noise mitigation strategy leveraging the statistical properties of the different noise terms most relevant in analogue hardware. Finally, we demonstrate the effectiveness of this combined approach for trained neural network classifying the MNIST handwritten digits, for which we achieve a 4-fold improvement of the output signal-to-noise ratio and increase the classification accuracy almost to the level of the noise-free network.

Neural and Evolutionary Computing,Machine Learning
Fundamental aspects of noise in analog-hardware neural networks

Nadezhda Semenova,Xavier Porte,Louis Andreoli,Maxime Jacquot,Laurent Larger,Daniel Brunner

DOI: https://doi.org/10.1063/1.5120824

2019-07-22

Abstract:We study and analyze the fundamental aspects of noise propagation in recurrent as well as deep, multi-layer networks. The main focus of our study are neural networks in analogue hardware, yet the methodology provides insight for networks in general. The system under study consists of noisy linear nodes, and we investigate the signal-to-noise ratio at the network's outputs which is the upper limit to such a system's computing accuracy. We consider additive and multiplicative noise which can be purely local as well as correlated across populations of neurons. This covers the chief internal-perturbations of hardware networks and noise amplitudes were obtained from a physically implemented recurrent neural network and therefore correspond to a real-world system. Analytic solutions agree exceptionally well with numerical data, enabling clear identification of the most critical components and aspects for noise management. Focusing on linear nodes isolates the impact of network connections and allows us to derive strategies for mitigating noise. Our work is the starting point in addressing this aspect of analogue neural networks, and our results identify notoriously sensitive points while simultaneously highlighting the robustness of such computational systems.

Emerging Technologies,Neural and Evolutionary Computing
How Do Efficient Coding Strategies Depend on Origins of Noise in Neural Circuits?

Braden A. W. Brinkman,Alison I. Weber,Fred Rieke,Eric Shea-Brown

DOI: https://doi.org/10.1371/journal.pcbi.1005150

2016-10-14

PLoS Computational Biology

Abstract:Neural circuits reliably encode and transmit signals despite the presence of noise at multiple stages of processing. The efficient coding hypothesis, a guiding principle in computational neuroscience, suggests that a neuron or population of neurons allocates its limited range of responses as efficiently as possible to best encode inputs while mitigating the effects of noise. Previous work on this question relies on specific assumptions about where noise enters a circuit, limiting the generality of the resulting conclusions. Here we systematically investigate how noise introduced at different stages of neural processing impacts optimal coding strategies. Using simulations and a flexible analytical approach, we show how these strategies depend on the strength of each noise source, revealing under what conditions the different noise sources have competing or complementary effects. We draw two primary conclusions: (1) differences in encoding strategies between sensory systems-or even adaptational changes in encoding properties within a given system-may be produced by changes in the structure or location of neural noise, and (2) characterization of both circuit nonlinearities as well as noise are necessary to evaluate whether a circuit is performing efficiently.

biochemical research methods,mathematical & computational biology
Walking Noise: On Layer-Specific Robustness of Neural Architectures against Noisy Computations and Associated Characteristic Learning Dynamics

Hendrik Borras,Bernhard Klein,Holger Fröning

2024-06-14

Abstract:Deep neural networks are extremely successful in various applications, however they exhibit high computational demands and energy consumption. This is exacerbated by stuttering technology scaling, prompting the need for novel approaches to handle increasingly complex neural architectures. At the same time, alternative computing technologies such as analog computing, which promise groundbreaking improvements in energy efficiency, are inevitably fraught with noise and inaccurate calculations. Such noisy computations are more energy efficient, and, given a fixed power budget, also more time efficient. However, like any kind of unsafe optimization, they require countermeasures to ensure functionally correct results. This work considers noisy computations in an abstract form, and gears to understand the implications of such noise on the accuracy of neural network classifiers as an exemplary workload. We propose a methodology called Walking Noise which injects layer-specific noise to measure the robustness and to provide insights on the learning dynamics. In more detail, we investigate the implications of additive, multiplicative and mixed noise for different classification tasks and model architectures. While noisy training significantly increases robustness for all noise types, we observe in particular that it results in increased weight magnitudes and thus inherently improves the signal-to-noise ratio for additive noise injection. Contrarily, training with multiplicative noise can lead to a form of self-binarization of the model parameters, leading to extreme robustness. We conclude with a discussion of the use of this methodology in practice, among others, discussing its use for tailored multi-execution in noisy environments.

Machine Learning,Hardware Architecture,Emerging Technologies
Improving Analog Neural Network Robustness: A Noise-Agnostic Approach with Explainable Regularizations

Alice Duque,Pedro Freire,Egor Manuylovich,Dmitrii Stoliarov,Jaroslaw Prilepsky,Sergei Turitsyn

2024-09-13

Abstract:This work tackles the critical challenge of mitigating "hardware noise" in deep analog neural networks, a major obstacle in advancing analog signal processing devices. We propose a comprehensive, hardware-agnostic solution to address both correlated and uncorrelated noise affecting the activation layers of deep neural models. The novelty of our approach lies in its ability to demystify the "black box" nature of noise-resilient networks by revealing the underlying mechanisms that reduce sensitivity to noise. In doing so, we introduce a new explainable regularization framework that harnesses these mechanisms to significantly enhance noise robustness in deep neural architectures.

Machine Learning,Artificial Intelligence,Optics
Activity-dependent Neuron Model for Noise Resistance

Yu Qi,Hanwen Wang,Rui Liu,Bian Wu,Yueming Wang,Gang Pan

DOI: https://doi.org/10.1016/j.neucom.2019.04.055

IF: 6

2019-01-01

Neurocomputing

Abstract:Activity-dependent plasticity plays an important role in biological neural network learning. Unlike the backpropagation-based learning in artificial neural networks that depends on supervised signals, biological neurons adjust themselves using their own historical behaviors as a clue to benefit information processing. Inspired by this biological neural mechanism, this paper proposes a novel neuron model, named Activity-Dependent Neuron (ADN). The basic idea of ADN is to add a gate in the neuron model to control its signal conduction capability, that is, the gate will facilitate transmission of important signals while suppress trivial signals. This idea can be achieved by self-tuning of activation behaviors. We also develop an iterative training algorithm, so that the ADNs can be smoothly incorporated into deep neural networks to jointly learn with the network weights. Experimental results found that the ADNs can efficiently improve the noise-resistant capability. Compared with the state-of-the-art, it is more robust to unforeseen noises. It does not need noise data for training.
Noise Helps Optimization Escape from Saddle Points in the Neural Dynamics.

Ying Fang,Zhaofei Yu,Feng Chen

2019-01-01

Abstract:Synaptic connectivity in the brain is thought to encode the long-term memory of an organism. But experimental data point to surprising ongoing fluctuations in synaptic activity. Assuming that the brain computation and plasticity can be understood as probabilistic inference, one of the essential roles of noise is to efficiently improve the performance of optimization in the form of stochastic gradient descent. The strict saddle condition for synaptic plasticity is deduced and under such condition noise can help escape from saddle points on high dimensional domains. The theoretical result explains the stochasticity of synapses and guides us how to make use of noise. Our simulation results manifest that in the learning and test phase, the accuracy of synaptic sampling is almost 20% higher than that without noise.
Noise driven neuromorphic tuned amplifier

Duccio Fanelli,Francesco Ginelli,Roberto Livi,Niccolò Zagli,Clement Zankoc

DOI: https://doi.org/10.1103/PhysRevE.96.062313

2017-07-07

Abstract:Living systems implement and execute an extraordinary plethora of computational tasks. The inherent degree of large scale coordination emerges as a global property, from the intricate sea of microscopic interactions. The brain, with its structural and functional architecture, represents an emblematic example of hierarchic self-organization: elementary units, the neurons, act much like instruments of an orchestra, which combine diverse timbres to create harmonious symphonies. Neurons come indeed in different types, varying in shapes, connections and electrical properties. They all team up to process external stimuli from a number of sources and integrate the information to yield, from neurons to mind, different cognitive faculties. Identifying the coarse grained modules that exert, from bottom to up, pivotal neuronal functions constitutes a goal of paramount importance. On the other hand, the brain and its unraveled secrets could inspire novel biomimetics technologies to adaptively handle complex problems. Here, we investigate the intertwined stochastic dynamics of two populations of excitatory and inhibitory units, arranged in a directed lattice. The endogenous noise seeds a coherent amplification across the chain generating giant oscillations with tunable frequencies, a process that the brain could exploit to enhance, and eventually encode, different signals. The system works as an out-equilibrium thermal device under stationary operating conditions, the associated entropy production rate being analytically quantified. The same scheme could be invoked to design a novel family of manmade detectors capable of reacting to spatially distributed low intensity alerts.

Disordered Systems and Neural Networks,Adaptation and Self-Organizing Systems,Pattern Formation and Solitons,Biological Physics
Training Recurrent Neural Networks against Noisy Computations during Inference

Minghai Qin,Dejan Vucinic

DOI: https://doi.org/10.48550/arXiv.1807.06555

2018-07-18

Abstract:We explore the robustness of recurrent neural networks when the computations within the network are noisy. One of the motivations for looking into this problem is to reduce the high power cost of conventional computing of neural network operations through the use of analog neuromorphic circuits. Traditional GPU/CPU-centered deep learning architectures exhibit bottlenecks in power-restricted applications, such as speech recognition in embedded systems. The use of specialized neuromorphic circuits, where analog signals passed through memory-cell arrays are sensed to accomplish matrix-vector multiplications, promises large power savings and speed gains but brings with it the problems of limited precision of computations and unavoidable analog noise. In this paper we propose a method, called {\em Deep Noise Injection training}, to train RNNs to obtain a set of weights/biases that is much more robust against noisy computation during inference. We explore several RNN architectures, such as vanilla RNN and long-short-term memories (LSTM), and show that after convergence of Deep Noise Injection training the set of trained weights/biases has more consistent performance over a wide range of noise powers entering the network during inference. Surprisingly, we find that Deep Noise Injection training improves overall performance of some networks even for numerically accurate inference.

Machine Learning
Noise Helps Optimization Escape from Saddle Points in the Synaptic Plasticity

Ying Fang,Zhaofei Yu,Feng Chen

DOI: https://doi.org/10.3389/fnins.2020.00343

IF: 4.3

2020-01-01

Frontiers in Neuroscience

Abstract:Numerous experimental studies suggest that noise is inherent in the human brain. However, the functional importance of noise remains unknown. n particular, from a computational perspective, such stochasticity is potentially harmful to brain function. In machine learning, a large number of saddle points are surrounded by high error plateaus and give the illusion of the existence of local minimum. As a result, being trapped in the saddle points can dramatically impair learning and adding noise will attack such saddle point problems in high-dimensional optimization, especially under the strict saddle condition. Motivated by these arguments, we propose one biologically plausible noise structure and demonstrate that noise can efficiently improve the optimization performance of spiking neural networks based on stochastic gradient descent. The strict saddle condition for synaptic plasticity is deduced, and under such conditions, noise can help optimization escape from saddle points on high dimensional domains. The theoretical results explain the stochasticity of synapses and guide us on how to make use of noise. In addition, we provide biological interpretations of proposed noise structures from two points: one based on the free energy principle in neuroscience and another based on observations of in vivo experiments. Our simulation results manifest that in the learning and test phase, the accuracy of synaptic sampling with noise is almost 20% higher than that without noise for synthesis dataset, and the gain in accuracy with/without noise is at least 10% for the MNIST and CIFAR-10 dataset. Our study provides a new learning framework for the brain and sheds new light on deep noisy spiking neural networks.
Noise-Aware Training of Neuromorphic Dynamic Device Networks

Luca Manneschi,Ian T. Vidamour,Kilian D. Stenning,Charles Swindells,Guru Venkat,David Griffin,Lai Gui,Daanish Sonawala,Denis Donskikh,Dana Hariga,Susan Stepney,Will R. Branford,Jack C. Gartside,Thomas Hayward,Matthew O. A. Ellis,Eleni Vasilaki

2024-10-29

Abstract:Physical computing has the potential to enable widespread embodied intelligence by leveraging the intrinsic dynamics of complex systems for efficient sensing, processing, and interaction. While individual devices provide basic data processing capabilities, networks of interconnected devices can perform more complex and varied tasks. However, designing networks to perform dynamic tasks is challenging without physical models and accurate quantification of device noise. We propose a novel, noise-aware methodology for training device networks using Neural Stochastic Differential Equations (Neural-SDEs) as differentiable digital twins, accurately capturing the dynamics and associated stochasticity of devices with intrinsic memory. Our approach employs backpropagation through time and cascade learning, allowing networks to effectively exploit the temporal properties of physical devices. We validate our method on diverse networks of spintronic devices across temporal classification and regression benchmarks. By decoupling the training of individual device models from network training, our method reduces the required training data and provides a robust framework for programming dynamical devices without relying on analytical descriptions of their dynamics.

Machine Learning,Artificial Intelligence,Emerging Technologies,Neural and Evolutionary Computing
A Blueprint for Precise and Fault-Tolerant Analog Neural Networks

Cansu Demirkiran,Lakshmi Nair,Darius Bunandar,Ajay Joshi

DOI: https://doi.org/10.1038/s41467-024-49324-8

2023-09-20

Abstract:Analog computing has reemerged as a promising avenue for accelerating deep neural networks (DNNs) due to its potential to overcome the energy efficiency and scalability challenges posed by traditional digital architectures. However, achieving high precision and DNN accuracy using such technologies is challenging, as high-precision data converters are costly and impractical. In this paper, we address this challenge by using the residue number system (RNS). RNS allows composing high-precision operations from multiple low-precision operations, thereby eliminating the information loss caused by the limited precision of the data converters. Our study demonstrates that analog accelerators utilizing the RNS-based approach can achieve ${\geq}99\%$ of FP32 accuracy for state-of-the-art DNN inference using data converters with only $6$-bit precision whereas a conventional analog core requires more than $8$-bit precision to achieve the same accuracy in the same DNNs. The reduced precision requirements imply that using RNS can reduce the energy consumption of analog accelerators by several orders of magnitude while maintaining the same throughput and precision. Our study extends this approach to DNN training, where we can efficiently train DNNs using $7$-bit integer arithmetic while achieving accuracy comparable to FP32 precision. Lastly, we present a fault-tolerant dataflow using redundant RNS error-correcting codes to protect the computation against noise and errors inherent within an analog accelerator.

Emerging Technologies,Artificial Intelligence,Hardware Architecture
Exploiting Inherent Error-Resiliency of Neuromorphic Computing to achieve Extreme Energy-Efficiency through Mixed-Signal Neurons

Baibhab Chatterjee,Priyadarshini Panda,Shovan Maity,Ayan Biswas,Kaushik Roy,Shreyas Sen

DOI: https://doi.org/10.48550/arXiv.1806.05141

2018-06-14

Abstract:Neuromorphic computing, inspired by the brain, promises extreme efficiency for certain classes of learning tasks, such as classification and pattern recognition. The performance and power consumption of neuromorphic computing depends heavily on the choice of the neuron architecture. Digital neurons (Dig-N) are conventionally known to be accurate and efficient at high speed, while suffering from high leakage currents from a large number of transistors in a large design. On the other hand, analog/mixed-signal neurons are prone to noise, variability and mismatch, but can lead to extremely low-power designs. In this work, we will analyze, compare and contrast existing neuron architectures with a proposed mixed-signal neuron (MS-N) in terms of performance, power and noise, thereby demonstrating the applicability of the proposed mixed-signal neuron for achieving extreme energy-efficiency in neuromorphic computing. The proposed MS-N is implemented in 65 nm CMOS technology and exhibits > 100X better energy-efficiency across all frequencies over two traditional digital neurons synthesized in the same technology node. We also demonstrate that the inherent error-resiliency of a fully connected or even convolutional neural network (CNN) can handle the noise as well as the manufacturing non-idealities of the MS-N up to certain degrees. Notably, a system-level implementation on MNIST datasets exhibits a worst-case increase in classification error by 2.1% when the integrated noise power in the bandwidth is ~ 0.1 uV2, along with +-3{\sigma} amount of variation and mismatch introduced in the transistor parameters for the proposed neuron with 8-bit precision.

Emerging Technologies,Artificial Intelligence,Neural and Evolutionary Computing
Overcoming device unreliability with continuous learning in a population coding based computing system

Alice Mizrahi,Julie Grollier,Damien Querlioz,M.D. Stiles

DOI: https://doi.org/10.1063/1.5042250

2018-06-01

Abstract:The brain, which uses redundancy and continuous learning to overcome the unreliability of its components, provides a promising path to building computing systems that are robust to the unreliability of their constituent nanodevices. In this work, we illustrate this path by a computing system based on population coding with magnetic tunnel junctions that implement both neurons and synaptic weights. We show that equipping such a system with continuous learning enables it to recover from the loss of neurons and makes it possible to use unreliable synaptic weights (i.e. low energy barrier magnetic memories). There is a tradeoff between power consumption and precision because low energy barrier memories consume less energy than high barrier ones. For a given precision, there is an optimal number of neurons and an optimal energy barrier for the weights that leads to minimum power consumption.

Emerging Technologies,Applied Physics
Noise-induced shallow circuits and absence of barren plateaus

Antonio Anna Mele,Armando Angrisani,Soumik Ghosh,Sumeet Khatri,Jens Eisert,Daniel Stilck França,Yihui Quek

2024-10-11

Abstract:Motivated by realistic hardware considerations of the pre-fault-tolerant era, we comprehensively study the impact of uncorrected noise on quantum circuits. We first show that any noise `truncates' most quantum circuits to effectively logarithmic depth, in the task of estimating observable expectation values. We then prove that quantum circuits under any non-unital noise exhibit lack of barren plateaus for cost functions composed of local observables. But, by leveraging the effective shallowness, we also design an efficient classical algorithm to estimate observable expectation values within any constant additive accuracy, with high probability over the choice of the circuit, in any circuit architecture. The runtime of the algorithm is independent of circuit depth, and for any inverse-polynomial target accuracy, it operates in polynomial time in the number of qubits for one-dimensional architectures and quasi-polynomial time for higher-dimensional ones. Taken together, our results showcase that, unless we carefully engineer the circuits to take advantage of the noise, it is unlikely that noisy quantum circuits are preferable over shallow quantum circuits for algorithms that output observable expectation value estimates, like many variational quantum machine learning proposals. Moreover, we anticipate that our work could provide valuable insights into the fundamental open question about the complexity of sampling from (possibly non-unital) noisy random circuits.

Quantum Physics,Other Condensed Matter,Mathematical Physics
Advancing Neuromorphic Computing: Mixed-Signal Design Techniques Leveraging Brain Code Units and Fundamental Code Units

Murat Isik,Sols Miziev,Wiktoria Pawlak,Newton Howard

2024-03-18

Abstract:This paper introduces a groundbreaking digital neuromorphic architecture that innovatively integrates Brain Code Unit (BCU) and Fundamental Code Unit (FCU) using mixedsignal design methodologies. Leveraging open-source datasets and the latest advances in materials science, our research focuses on enhancing the computational efficiency, accuracy, and adaptability of neuromorphic systems. The core of our approach lies in harmonizing the precision and scalability of digital systems with the robustness and energy efficiency of analog processing. Through experimentation, we demonstrate the effectiveness of our system across various metrics. The BCU achieved an accuracy of 88.0% and a power efficiency of 20.0 GOP/s/W, while the FCU recorded an accuracy of 86.5% and a power efficiency of 18.5 GOP/s/W. Our mixed-signal design approach significantly improved latency and throughput, achieving a latency as low as 0.75 ms and throughput up to 213 TOP/s. These results firmly establish the potential of our architecture in neuromorphic computing, providing a solid foundation for future developments in this domain. Our study underscores the feasibility of mixedsignal neuromorphic systems and their promise in advancing the field, particularly in applications requiring high efficiency and adaptability

Hardware Architecture,Machine Learning,Neural and Evolutionary Computing

Overcoming the noise in neural computing

Noisy Machines: Understanding Noisy Neural Networks and Enhancing Robustness to Analog Hardware Errors Using Distillation

Exploiting Noise As a Resource for Computation and Learning in Spiking Neural Networks

Understanding and mitigating noise in trained deep neural networks

Noisy Computations during Inference: Harmful or Helpful?

Noise mitigation strategies in physical feedforward neural networks

Fundamental aspects of noise in analog-hardware neural networks

How Do Efficient Coding Strategies Depend on Origins of Noise in Neural Circuits?

Walking Noise: On Layer-Specific Robustness of Neural Architectures against Noisy Computations and Associated Characteristic Learning Dynamics

Improving Analog Neural Network Robustness: A Noise-Agnostic Approach with Explainable Regularizations

Activity-dependent Neuron Model for Noise Resistance

Noise Helps Optimization Escape from Saddle Points in the Neural Dynamics.

Noise driven neuromorphic tuned amplifier

Training Recurrent Neural Networks against Noisy Computations during Inference

Noise Helps Optimization Escape from Saddle Points in the Synaptic Plasticity

Noise-Aware Training of Neuromorphic Dynamic Device Networks

A Blueprint for Precise and Fault-Tolerant Analog Neural Networks

Exploiting Inherent Error-Resiliency of Neuromorphic Computing to achieve Extreme Energy-Efficiency through Mixed-Signal Neurons

Overcoming device unreliability with continuous learning in a population coding based computing system

Noise-induced shallow circuits and absence of barren plateaus

Advancing Neuromorphic Computing: Mixed-Signal Design Techniques Leveraging Brain Code Units and Fundamental Code Units