Abstract:Memristive crossbars have become a popular means for realizing unsupervised and supervised learning techniques. In previous neuromorphic architectures with leaky integrate-and-fire neurons, the crossbar itself has been separated from the neuron capacitors to preserve mathematical rigor. In this paper, we sought to design a simplified sparse coding circuit without this restriction, resulting in a fast circuit that approximated a sparse coding operation at a minimal loss in accuracy. We showed that connecting the neurons directly to the crossbar resulted in a more energy-efficient sparse coding architecture and alleviated the need to prenormalize receptive fields. This paper provides derivations for the design of such a network, named the simple spiking locally competitive algorithm, as well as CMOS designs and results on the CIFAR and MNIST data sets. Compared to a nonspiking, nonapproximate model which scored 33% on CIFAR-10 with a single-layer classifier, this hardware scored 32% accuracy. When used with a state-of-the-art deep learning classifier, the nonspiking model achieved 82% and our simplified, spiking model achieved 80% while compressing the input data by 92%. Compared to a previously proposed spiking model, our proposed hardware consumed 99% less energy to do the same work at 21 × the throughput. Accuracy held out with online learning to a write variance of 3%, suitable for the often reported 4-bit resolution required for neuromorphic algorithms, with offline learning to a write variance of 27%, and with read variance to 40%. The proposed architecture's excellent accuracy, throughput, and significantly lower energy usage demonstrate the utility of our innovations.

Sparse coding with memristor networks

Adaptive Sparse Coding Based on Memristive Neural Network with Applications

Memristive neural network circuit design based on locally competitive algorithm for sparse coding application

Single Image Super-Resolution Via the Implementation of the Hardware-Friendly Sparse Coding

A Brain-Inspired In-Memory Computing System for Neuronal Communication Via Memristive Circuits.

Fast and Accurate Sparse Coding of Visual Stimuli With a Simple, Ultralow-Energy Spiking Architecture

Forward Stagewise Regression with Multilevel Memristor for Sparse Coding

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

A CMOS spiking neuron for dense memristor-synapse connectivity for brain-inspired computing

Memristor-Based Neuromorphic System With Content Addressable Memory Structure

Memristors: Associative Learning with Temporal Contiguity in a Memristive Circuit for Large‐Scale Neuromorphic Networks (Adv. Electron. Mater. 8/2015)

Spiking-based matrix computation by leveraging memristor crossbar array

Memristor‐Based Analog Computation and Neural Network Classification with a Dot Product Engine

Sparse Realization in Unreliable Spin-Transfer-Torque RAM for Convolutional Neural Network

Homogeneous Spiking Neuromorphic System for Real-World Pattern Recognition

Long short-term memory networks in memristor crossbar arrays

Modeling and Analysis of Spike Signal Sequence for Memristor Crossbar Array in Neuromorphic Chips

Fully Memristive SNNs with Temporal Coding for Fast and Low-power Edge Computing

Associative Learning with Temporal Contiguity in a Memristive Circuit for Large‐Scale Neuromorphic Networks

Memristor Neural Network Design

On-chip Supervised Learning Rule for Ultra High Density Neural Crossbar Using Memristor for Synapse and Neuron