Code for Edge Learning Using a Fully Integrated Neuro-Inspired Memristor Chip

W. Zhang,Peng Yao,Bin Gao,Huaqiang Wu

DOI: https://doi.org/10.5281/zenodo.8151757

2023-01-01

Abstract:Code for Edge Learning Using a Fully Integrated Neuro-Inspired Memristor Chip. These codes can also be accessed through GitHub: https://github.com/Tsinghua-LEMON-Lab/software_edge_learning_memristor_chip.git.

What problem does this paper attempt to address?

A Fully-Integrated Memristor Chip for Edge Learning

Yanhong Zhang,Liang Chu,Wenjun Li

DOI: https://doi.org/10.1007/s40820-024-01368-7

IF: 26.6

2024-04-03

Nano-Micro Letters

Abstract:Highlights The fully-integrated memristor chip for edge learning provides a solid foundation for neural network computation. The fully-integrated memristor chip enables efficient object recognition in noisy backgrounds while minimizing energy consumption. The computing-in-memory chip represents an innovative and interdisciplinary technology that extends beyond multiple research domains.

materials science, multidisciplinary,physics, applied,nanoscience & nanotechnology
Edge learning using a fully integrated neuro-inspired memristor chip

Wenbin Zhang,Peng Yao,Bin Gao,Qi Liu,Dong Wu,Qingtian Zhang,Yuankun Li,Qi Qin,Jiaming Li,Zhenhua Zhu,Yi Cai,Dabin Wu,Jianshi Tang,He Qian,Yu Wang,Huaqiang Wu

DOI: https://doi.org/10.1126/science.ade3483

IF: 56.9

2023-09-15

Science

Abstract:Learning is highly important for edge intelligence devices to adapt to different application scenes and owners. Current technologies for training neural networks require moving massive amounts of data between computing and memory units, which hinders the implementation of learning on edge devices. We developed a fully integrated memristor chip with the improvement learning ability and low energy cost. The schemes in the STELLAR architecture, including its learning algorithm, hardware realization, and parallel conductance tuning scheme, are general approaches that facilitate on-chip learning by using a memristor crossbar array, regardless of the type of memristor device. Tasks executed in this study included motion control, image classification, and speech recognition.

multidisciplinary sciences
An 8-Bit in Resistive Memory Computing Core with Regulated Passive Neuron and Bitline Weight Mapping

Yewei Zhang,Kejie Huang,Rui Xiao,Bo Wang,Yanfeng Xu,Jicong Fan,Haibin Shen

DOI: https://doi.org/10.1109/tvlsi.2022.3140395

2022-01-01

IEEE Transactions on Very Large Scale Integration (VLSI) Systems

Abstract:The rapid development of artificial intelligence (AI) and Internet of Things (IoT) increase the requirement for edge computing with low power and relatively high processing speed devices. The computing-in-memory (CIM) schemes based on emerging resistive nonvolatile memory (NVM) show great potential in reducing the power consumption for AI computing. However, the inconsistency of the NVM may significantly degenerate the performance of the neural network. In this article, we propose a low power resistive RAM (RRAM)-based CIM core to not only achieve high computing efficiency but also greatly enhance the robustness by bit line (BL) regulator and BL weight mapping algorithm. The simulation results show that the power consumption of our proposed 8-bit CIM core is only 12.6 mW ( $256\times 256$ at 8b). The spurious-free dynamic range (SFDR) and signal to noise and distortion ratio (SNDR) of the CIM core achieve 62.64 and 45.92 dB, respectively. The proposed BL weight mapping scheme improves the top-1 accuracy by 2.46% and 3.47% for AlexNet and VGG16 on ImageNet Large Scale Visual Recognition Competition 2012 (ILSVRC 2012) in 8-bit mode, respectively.
Transforming edge hardware with in situ learning features

Peng Yao,Bin Gao,Huaqiang Wu

DOI: https://doi.org/10.1038/s44287-024-00031-y

2024-01-01

Abstract:Memristor devices have shown notable superiority in the realm of neuromorphic computing chips, particularly in artificial intelligence (AI) inference tasks. Researchers are now grappling with the intricacies of incorporating in situ learning capabilities into memristor-based chips, paving the way for more powerful edge intelligence.
Hardware Implementation of Edge Neural Network Computing for Sensor With Memristors Based on Single-Crystalline LiNbO 3 Thin Film

Xinqiang Pan,Wen Luo,Yao Shuai,Shaogang Hu,Wenbo Luo,Guanchao Qiao,Tong Zhou,Jiejun Wang,Qin Xie,Shitian Huang,Yang Liu,Chuangui Wu,Wanli Zhang

DOI: https://doi.org/10.1109/jsen.2023.3248123

IF: 4.3

2023-04-18

IEEE Sensors Journal

Abstract:With the development of artificial intelligent IoT (AIoT) and dramatic increase of the amount of data, conventional architecture, in which all the data collected by sensors are sent to the data center for processing and computing, suffered from heavy computing load in the data center, latency, and high energy consumption. Edge neural network computing at the sensor terminal is demanding. Based on the advantages of memristor in nature co-location of memory and computing, high computing parallelism, low energy consumption, and miniaturization potential, we proposed to take use of memristors to conduct edge neural network computing at sensor and realize the integration of sensing memory computing. Also, in order to solve the problems caused by resistance states variation and device-to-device variation without transistor connected in series, highly uniform memristors based on single-crystalline LiNbO3 (LN) thin film with two stable resistance states were fabricated and utilized to realize binarized neural networks computing, and the coupling between the pressure sensor output signal with the input of the memristor array has been built. The hardware implementation of memristor-based edge neural network computing on the signals of pressure sensor array has been realized. With the memristor-based edge neural network computing, recognition of three letters ("V," "Z," and "T") wrote on the pressure sensor array has been realized.

engineering, electrical & electronic,instruments & instrumentation,physics, applied
Fully Memristive SNNs with Temporal Coding for Fast and Low-power Edge Computing

Xumeng Zhang,Zuheng Wu,Jikai Lu,Jinsong Wei,Jian Lu,Jiaxue Zhu,Jie Qiu,Rui Wang,Kaihua Lou,Yongzhou Wang,Tuo Shi,Chunmeng Dou,Dashan Shang,Qi Liu,Ming Liu

DOI: https://doi.org/10.1109/iedm13553.2020.9371937

2020-01-01

Abstract:SNNs with temporal coding (TC), inspired by the human visual system, have a powerful ability to enable fast and low-power neuromorphic computing. Memristive devices show excellent performance on emulating spiking neurons and synapses in hardware. However, the neuron circuits used for implementing a fully memristive TC SNN are absent. In this work, for the first time, we demonstrate a LIF neuron based on a NbO x device to meet the requirements for the hardware implementation of TC SNNs. The neuron fires at most one spike within an inference window, and its spiking latency inverse to the input current intensity. Using such a neuron, we further experimentally demonstrated a fully memristive TC SNN (256 × 5) to recognize the Olivetti face patterns. Attributing to the one-spike scheme, the TC SNN achieves a sparser spiking number (~ 72 × reductions), faster inference speed (> 1.5 × improvement), lower power (~ 53 × reductions) than what happens in rate-coding SNNs.
Hardware Implementation of Edge Neural Network Computing for Sensor with Memristors Based on Single-Crystalline LiNbO3 Thin Film

Xinqiang Pan,Wen Luo,Yao Shuai,Shaogang Hu,Wenbo Luo,Guanchao Qiao,Tong Zhou,Jiejun Wang,Qin Xie,Shitian Huang,Yang Liu,Chuangui Wu,Wanli Zhang

DOI: https://doi.org/10.1109/jsen.2023.3248123

IF: 4.3

2023-01-01

IEEE Sensors Journal

Abstract:With the development of artificial intelligent IoT (AIoT) and dramatic increase of the amount of data, conventional architecture, in which all the data collected by sensors are sent to the data center for processing and computing, suffered from heavy computing load in the data center, latency, and high energy consumption. Edge neural network computing at the sensor terminal is demanding. Based on the advantages of memristor in nature co-location of memory and computing, high computing parallelism, low energy consumption, and miniaturization potential, we proposed to take use of memristors to conduct edge neural network computing at sensor and realize the integration of sensing memory computing. Also, in order to solve the problems caused by resistance states variation and device-to-device variation without transistor connected in series, highly uniform memristors based on single-crystalline LiNbO3 (LN) thin film with two stable resistance states were fabricated and utilized to realize binarized neural networks computing, and the coupling between the pressure sensor output signal with the input of the memristor array has been built. The hardware implementation of memristor-based edge neural network computing on the signals of pressure sensor array has been realized. With the memristor-based edge neural network computing, recognition of three letters (“V,” “Z,” and “T”) wrote on the pressure sensor array has been realized.
Fully memristive spiking-neuron learning framework and its applications on pattern recognition and edge detection

Zhiri Tang,Yanhua Chen,Shizhuo Ye,Ruihan Hu,Hao Wang,Jin He,Qijun Huang,Sheng Chang

DOI: https://doi.org/10.1016/j.neucom.2020.04.012

IF: 6

2020-08-01

Neurocomputing

Abstract:<p>Fully memristive neuron learning framework, which uses drift and diffusion memristor models to build an artificial neuron structure, becomes a hot topic recently with the development of memristor. However, some other devices like resistor or capacitor are still necessary in recent works of fully memristive learning framework. Theoretically, if one neuron is built by memristors only, the technique process will be simpler and learning framework will be more like biological brain. In this paper, a fully memristive spiking-neuron learning framework is introduced, in which a neuron structure is just built of one drift and one diffusion memristive models and spikes are used as transmission signals. The learning framework and spiking coding mode are simple and direct without any complicated calculation on hardware. To verify its merits, a feedforward neural network for pattern recognition and a cellular neural network for edge detection are designed. Experimental results show that compared to other memristive neural networks, processing speed of the proposed framework is very high, and the hardware resource is saved in pattern recognition. Further, due to the dynamic filtering function of diffusion memristor model in our learning framework, its peak signal noise ratio (PSNR) is much higher than traditional algorithms in edge detection.</p>

computer science, artificial intelligence
NeuroPack: An Algorithm-Level Python-Based Simulator for Memristor-Empowered Neuro-Inspired Computing

Jinqi Huang,Spyros Stathopoulos,Alexantrou Serb,Themis Prodromakis

DOI: https://doi.org/10.3389/fnano.2022.851856

2022-04-20

Frontiers in Nanotechnology

Abstract:Emerging two-terminal nanoscale memory devices, known as memristors, have demonstrated great potential for implementing energy-efficient neuro-inspired computing architectures over the past decade. As a result, a wide range of technologies have been developed that, in turn, are described via distinct empirical models. This diversity of technologies requires the establishment of versatile tools that can enable designers to translate memristors’ attributes in novel neuro-inspired topologies. In this study, we present NeuroPack, a modular, algorithm-level Python-based simulation platform that can support studies of memristor neuro-inspired architectures for performing online learning or offline classification. The NeuroPack environment is designed with versatility being central, allowing the user to choose from a variety of neuron models, learning rules, and memristor models. Its hierarchical structure empowers NeuroPack to predict any memristor state changes and the corresponding neural network behavior across a variety of design decisions and user parameter options. The use of NeuroPack is demonstrated herein via an application example of performing handwritten digit classification with the MNIST dataset and an existing empirical model for metal-oxide memristors.
Neuromorphic Extreme Learning Machines with Bimodal Memristive Synapses

Zhekang Dong,Chun Sing Lai,Zhaowei Zhang,Donglian Qi,Mingyu Gao,Shukai Duan

DOI: https://doi.org/10.1016/j.neucom.2021.04.049

IF: 6

2021-01-01

Neurocomputing

Abstract:The biology-inspired intelligent computing system for the neuromorphic hardware implementation is useful in high-speed parallel information processing. However, the traditional Von Neumann computer architecture and the unsatisfactory signal transmission approach have jointly limited the overall performance of the specific hardware implementation. In this paper, a compact extreme learning machine (ELM) architecture synthesized with the spintronic memristor-based synaptic circuit, the biasing circuit, and the activation function circuit is presented. Notably, due to the threshold characteristic of the memristive device, the synaptic circuit has a bimodal behavior. Namely, it is capable to provide the constant and adjustable network weights between the adjacent layers in the ELM. Furthermore, two major limitations (process variations and sneak path issue) are taken into account for the detailed robustness analysis of the whole network. Finally, the entire scheme is verified with case studies in single image super-resolution (SR) reconstruction.
End-to-end Edge Detection on Self-rectifying Resistive Memory Array

Shurong Pan,Wenhan Liu,Hao Wang,Qijun Huang,Jin He,Ruiqing Cheng,Sheng Chang

DOI: https://doi.org/10.1039/d4tc03155e

IF: 6.4

2024-10-19

Journal of Materials Chemistry C

Abstract:In recent years, memristor that can integrate storage and computing has gained some attention in image processing. In this study, edge detection for noisy images is realized in a single-step manner by a new Gauss-Laplace operator in a circuit with a memristor array. It ensures the edge detection end-to-end and reduces hardware buffer consumption compared to the traditional two-step methods. Conductance values from our experimentally fabricated Cu2Te memory device are used in the circuit design, which mitigates the gap between theory and real-world device arrays. Furthermore, an interesting finding is that the edge detection results in the circuit are closer to the ideal ones after reducing the grayscale from 256 to 16, which means fewer input voltage levels and a simpler input circuit are possible. It is hoped this work can help design memristor array circuits for image processing.

materials science, multidisciplinary,physics, applied
Energy‐Efficient Memristive Euclidean Distance Engine for Brain‐Inspired Competitive Learning

Houji Zhou,Jia Chen,Yinan Wang,Sen Liu,Yi Li,Qingjiang Li,Qi Liu,Zhongrui Wang,Yuhui He,Hui Xu,Xiangshui Miao

DOI: https://doi.org/10.1002/aisy.202100114

IF: 7.298

2021-01-01

Advanced Intelligent Systems

Abstract:Inspired by competitive rules of the nature, competitive learning contributes to the specialization of the human brain and the general creativity of mankind. However, the construction of hardware competitive learning neural network still faces great challenges due to the lack of an accurate distance computation method and a self‐adaptive in situ training scheme. Herein, a fully memristive Euclidean distance (ED) engine based on analog multiply‐accumulate operation in a 32 × 32 TiN/TaO x /HfO x /TiN 1T1R array is demonstrated. The dual‐layer devices perform multilevel modulation under the target‐aware programming method with excellent read linearity in a dynamic range of 10–100 μS. The ED calculation is verified experimentally on a test board with an O (1) temporal complexity. Furthermore, in situ training and offline inference schemes for competitive learning, based on the ED engine, are developed and the simulated results show comparable success rates with those obtained by the CPU‐based software. Compared with a state‐of‐the‐art RTX6000 GPU (0.5 TOPS W −1 ), the energy efficiency of competitive learning models on ED engines can yield 100× improvements by utilizing optimized memristive devices.
A Memristor-Based Learning Engine for Synaptic Trace-Based Online Learning.

Deyu Wang,Jiawei Xu,Feng Li,Lianhao Zhang,Chengwei Cao,Dimitrios Stathis,Anders Lansner,Ahmed Hemani,Li-Rong Zheng,Zhuo Zou

DOI: https://doi.org/10.1109/tbcas.2023.3291021

2023-01-01

IEEE Transactions on Biomedical Circuits and Systems

Abstract:The memristor has been extensively used to facilitate the synaptic online learning of brain-inspired spiking neural networks (SNNs). However, the current memristor-based work can not support the widely used yet sophisticated trace-based learning rules, including the trace-based Spike-Timing-Dependent Plasticity (STDP) and the Bayesian Confidence Propagation Neural Network (BCPNN) learning rules. This paper proposes a learning engine to implement trace-based online learning, consisting of memristor-based blocks and analog computing blocks. The memristor is used to mimic the synaptic trace dynamics by exploiting the nonlinear physical property of the device. The analog computing blocks are used for the addition, multiplication, logarithmic and integral operations. By organizing these building blocks, a reconfigurable learning engine is architected and realized to simulate the STDP and BCPNN online learning rules, using memristors and 180 nm analog CMOS technology. The results show that the proposed learning engine can achieve energy consumption of 10.61 pJ and 51.49 pJ per synaptic update for the STDP and BCPNN learning rules, respectively, with a 147.03× and 93.61× reduction compared to the 180 nm ASIC counterparts, and also a 9.39× and 5.63× reduction compared to the 40 nm ASIC counterparts. Compared with the state-of-the-art work of Loihi and eBrainII, the learning engine can reduce the energy per synaptic update by 11.31× and 13.13× for trace-based STDP and BCPNN learning rules, respectively.
N‐P Reconfigurable Dual‐Mode Memtransistors for Compact Bio‐Inspired Feature Extractor with Inhibitory‐Excitatory Spiking Capability

Jin Feng Leong,Zihang Fang,Maheswari Sivan,Jieming Pan,Baoshan Tang,Evgeny Zamburg,Aaron V‐Y Thean

DOI: https://doi.org/10.1002/adfm.202302949

IF: 19

2023-07-13

Advanced Functional Materials

Abstract:A dual mode, reconfigurable spike timing‐dependent plasticity (STDP)/anti‐STDP vacancy‐modulated floating‐gate memtransistor is introduced into a competitive learning based spiking neural network architecture with significant savings in device count (≈27%) in peripheral circuits through a device‐system co‐design approach. This bio‐inspired architecture can accelerate convergence in recognition tasks of up to 5.7 times compared to prevalent winner‐takes‐all approaches in competitive learning implementation. Competitive‐learning‐based spiking neural networks are capable of rapid, highly accurate pattern recognition with minimal data through denoising mechanisms provide by adaptive interneuron inhibition. However, hardware implementations of such networks are currently area‐inefficient due to the high device count require to execute dual excitatory‐inhibitory synapses. To mitigate this, n‐/p‐ reconfigurable tungsten diselenide memtransistors is introduced that can execute excitatory and inhibitory synapses in a highly compact bio‐inspired feature extractor hardware architecture. The reconfigurability is realized through a dual mode memory device with a flash‐memory‐like floating‐gate for n‐/p‐ programing and a memristor‐like selenium vacancy‐based resistive switching that varies in memristive output with majority carrier modulation. Through a device‐system codesign, an effective 27% device count reduction in the peripheral circuits is achieved , which ameliorates circuit component congestion and circuit complexity. Compared to the prevalent winner‐takes‐all approach, the proposed machine learning with adaptive interneuron inhibition achieves high‐accuracy convergence with up to five times smaller training dataset. This accelerated learning can potentially enable edge‐artificial intelligence (AI) processors capable of ultra‐low‐energy training with limited data.

materials science, multidisciplinary,chemistry, physical,physics, applied, condensed matter,nanoscience & nanotechnology
Fully Hardware Memristive Neuromorphic Computing Enabled by the Integration of Trainable Dendritic Neurons and High‐Density RRAM Chip

Zhen Yang,Wenshuo Yue,Chang Liu,Yaoyu Tao,Pek Jun Tiw,Longhao Yan,Yuxiang Yang,Teng Zhang,Bingjie Dang,Keqin Liu,Xiaodong He,Yongqin Wu,Weihai Bu,Kai Zheng,Jin Kang,Ru Huang,Yuchao Yang

DOI: https://doi.org/10.1002/adfm.202405618

IF: 19

2024-05-15

Advanced Functional Materials

Abstract:An efficient dendrite‐function‐like negative‐differential‐resistance (NDR) neuron is proposed for the first time. By co‐integrating electrochemicalrandom‐access memory (ECRAM) and ionic regulation, the non‐ideality of devicesis mitigated and it can also be trained to enhance neural network performancefor edge learning. Finally, the NDR neuron can work synergistically with 1T1R arrays to achieve full hardware implementation of neural networks. Computing‐in‐memory (CIM) architecture inspired by the hierarchy of human brain is proposed to resolve the von Neumann bottleneck and boost acceleration of artificial intelligence. Whereas remarkable progress has been achieved for CIM, making further improvements in CIM performance is becoming increasingly challenging, which is mainly caused by the disparity between rapid evolution of synaptic arrays and relatively slow progress in building efficient neuronal devices. Specifically, dedicated efforts are required toward developments of more advanced activation units in terms of both optimized algorithms and innovative hardware implementations. Here a novel bio‐inspired dendrite function‐like neuron based on negative‐differential‐resistance (NDR) behavior is reported and experimentally demonstrates this design as a more efficient neuron. By integrating electrochemical random‐access memory (ECRAM) with ionic regulation, the tunable NDR neuron can be trained to enhance neural network performances. Furthermore, based on a high‐density RRAM chip, fully hardware implementation of CIM is experimentally demonstrated by integrating NDR neuron devices with only a 1.03% accuracy loss. This work provides 516 × and 1.3 × 105 × improvements on LAE (Latency‐Area‐Energy) property, compared to the digital and analog CMOS activation circuits, respectively. With device‐algorithm co‐optimization, this work proposes a compact and energy‐efficient solution that pushes CIM‐based neuromorphic computing into a new paradigm.

materials science, multidisciplinary,chemistry, physical,physics, applied, condensed matter,nanoscience & nanotechnology
Memristive Extreme Learning Machine: A Neuromorphic Implementation

Lu Zhang,Hong Cheng,Huanghuang Liang,Yang Zhao,Xinqiang Pan,Yuansheng Luo,Hongliang Guo,Yao Shuai

DOI: https://doi.org/10.1007/978-3-030-01520-6_11

2018-01-01

Abstract:Neuromorphic computation has been a hot research area over the past few years. Memristor, as one of the neuromorphic computation materials memorizes the conductance value and is able to adapt it according to changing voltages. This paper pioneers a neuromorphic computing paradigm implementation (through memristor) for Extreme Learning Machine (ELM), which is one of most popular machine learning methods. By simulating the biological synapses with memristors and combining the memory property of memristor with high-efficient processing ability in ELM, a three-layer ELM model for classification is constructed. We represent the ELM network weights through memristive conductance values. The conductance values (network weights) are updated through tuning the voltages. Experimental results over the Iris dataset show that the memristor-based ELM achieves the same level performance as the one implemented via traditional software, and exhibits great potential that ELM can be implemented in neromorphic computation paradigms.
In-Memory Computing With Memristor Arrays

Can Li,Daniel Belkin,Yunning Li,Peng Yan,Miao Hu,Ning Ge,Hao Jiang,Eric Montgomery,Peng Lin,Zhongrui Wang,John Paul Strachan,Mark Barnell,Qing Wu,R. Stanley Williams,J. Joshua Yang,Qiangfei Xia

DOI: https://doi.org/10.1109/IMW.2018.8388838

2018-01-01

Abstract:Memristors with tunable non-volatile resistance states offer the potential for in-memory computing that mitigates the von-Neumann bottleneck. We build a large scale memristor array by integrating a transistor array with Ta/HfO2 memristors that have stable multilevel resistance states and linear IV characteristic. With off-chip peripheral driving circuits, the memristor chip is capable of high-precision analog computing and online learning. We demonstrate a weight-update scheme that provides linear and symmetric potentiation and depression with no more than two pulses for each cell. We train the array as a single-layer fully-connected feedforward neural network for the WDBC data base and achieve 98% classification accuracy. We further partition the array into a two-layer network, which achieves 91.71% classification accuracy for MNIST database experimentally. The system demonstrates high defect tolerance and excellent speed-energy efficiency.
The Ouroboros of Memristors: Neural Networks Facilitating Memristor Programming

Zhenming Yu,Ming-Jay Yang,Jan Finkbeiner,Sebastian Siegel,John Paul Strachan,Emre Neftci

DOI: https://doi.org/10.48550/arXiv.2403.06712

2024-03-11

Emerging Technologies

Abstract:Memristive devices hold promise to improve the scale and efficiency of machine learning and neuromorphic hardware, thanks to their compact size, low power consumption, and the ability to perform matrix multiplications in constant time. However, on-chip training with memristor arrays still faces challenges, including device-to-device and cycle-to-cycle variations, switching non-linearity, and especially SET and RESET asymmetry. To combat device non-linearity and asymmetry, we propose to program memristors by harnessing neural networks that map desired conductance updates to the required pulse times. With our method, approximately 95% of devices can be programmed within a relative percentage difference of +-50% from the target conductance after just one attempt. Our approach substantially reduces memristor programming delays compared to traditional write-and-verify methods, presenting an advantageous solution for on-chip training scenarios. Furthermore, our proposed neural network can be accelerated by memristor arrays upon deployment, providing assistance while reducing hardware overhead compared with previous works. This work contributes significantly to the practical application of memristors, particularly in reducing delays in memristor programming. It also envisions the future development of memristor-based machine learning accelerators.
A Neuromorphic Core Based on Threshold Switching Memristor with Asynchronous Address Event Representation Circuits

Zhejiang Lab,Zhang Jilin,Zhang Xumeng,Wu Zuheng,Wang Rui,Chan Mansun,Chen Hong

DOI: https://doi.org/10.1007/s11432-020-3203-0

2021-01-01

Science China Information Sciences

Abstract:The full memristive network hardware features high density and excellent scalability. However, recent researches on the full memristive network have been limited to a single-layer network, due to the lack of effective and flexible communication between neurons. In this design, we demonstrate a neuromorphic core based on Ag/SiO2/Au threshold switching memristor, which has built-in asynchronous address event representation (AER) circuits to provide flexible communication between neurons. Since temporally sparse spikes are the medium of communication between neurons, the AER circuits are designed to transmit spikes serially which have been encoded with neurons’ addresses before transmission. With the asynchronous circuits design, the AER circuits will detect neurons’ output in real-time. To test the performance of the neuromorphic core, we designed a behavioral simulator for the neuromporphic core to simulate the liquid state machine (LSM) network, which achieves a 100% recognition rate in the free spoken digital dataset. The simulation results show that the neuromorphic core obtains 35 times higher performance than the CPU and 111 times higher energy efficiency than the GPU.
In Situ Training of Feed-Forward and Recurrent Convolutional Memristor Networks

Zhongrui Wang,Can Li,Peng Lin,Mingyi Rao,Yongyang Nie,Wenhao Song,Qinru Qiu,Yunning Li,Peng Yan,John Paul Strachan,Ning Ge,Nathan McDonald,Qing Wu,Miao Hu,Huaqiang Wu,R. Stanley Williams,Qiangfei Xia,J. Joshua Yang

DOI: https://doi.org/10.1038/s42256-019-0089-1

IF: 23.8

2019-01-01

Nature Machine Intelligence

Abstract:The explosive growth of machine learning is largely due to the recent advancements in hardware and architecture. The engineering of network structures, taking advantage of the spatial or temporal translational isometry of patterns, naturally leads to bio-inspired, shared-weight structures such as convolutional neural networks, which have markedly reduced the number of free parameters. State-of-the-art microarchitectures commonly rely on weight-sharing techniques, but still suffer from the von Neumann bottleneck of transistor-based platforms. Here, we experimentally demonstrate the in situ training of a five-level convolutional neural network that self-adapts to non-idealities of the one-transistor one-memristor array to classify the MNIST dataset, achieving similar accuracy to the memristor-based multilayer perceptron with a reduction in trainable parameters of ~75% owing to the shared weights. In addition, the memristors encoded both spatial and temporal translational invariance simultaneously in a convolutional long short-term memory network—a memristor-based neural network with intrinsic 3D input processing—which was trained in situ to classify a synthetic MNIST sequence dataset using just 850 weights. These proof-of-principle demonstrations combine the architectural advantages of weight sharing and the area/energy efficiency boost of the memristors, paving the way to future edge artificial intelligence.

Code for Edge Learning Using a Fully Integrated Neuro-Inspired Memristor Chip

A Fully-Integrated Memristor Chip for Edge Learning

Edge learning using a fully integrated neuro-inspired memristor chip

An 8-Bit in Resistive Memory Computing Core with Regulated Passive Neuron and Bitline Weight Mapping

Transforming edge hardware with in situ learning features

Hardware Implementation of Edge Neural Network Computing for Sensor With Memristors Based on Single-Crystalline LiNbO 3 Thin Film

Fully Memristive SNNs with Temporal Coding for Fast and Low-power Edge Computing

Hardware Implementation of Edge Neural Network Computing for Sensor with Memristors Based on Single-Crystalline LiNbO3 Thin Film

Fully memristive spiking-neuron learning framework and its applications on pattern recognition and edge detection

NeuroPack: An Algorithm-Level Python-Based Simulator for Memristor-Empowered Neuro-Inspired Computing

Neuromorphic Extreme Learning Machines with Bimodal Memristive Synapses

End-to-end Edge Detection on Self-rectifying Resistive Memory Array

Energy‐Efficient Memristive Euclidean Distance Engine for Brain‐Inspired Competitive Learning

A Memristor-Based Learning Engine for Synaptic Trace-Based Online Learning.

N‐P Reconfigurable Dual‐Mode Memtransistors for Compact Bio‐Inspired Feature Extractor with Inhibitory‐Excitatory Spiking Capability

Fully Hardware Memristive Neuromorphic Computing Enabled by the Integration of Trainable Dendritic Neurons and High‐Density RRAM Chip

Memristive Extreme Learning Machine: A Neuromorphic Implementation

In-Memory Computing With Memristor Arrays

The Ouroboros of Memristors: Neural Networks Facilitating Memristor Programming

A Neuromorphic Core Based on Threshold Switching Memristor with Asynchronous Address Event Representation Circuits

In Situ Training of Feed-Forward and Recurrent Convolutional Memristor Networks