Abstract:The unprecedented growth in the field of machine learning has led to the development of deep neuromorphic networks trained on labelled dataset with capability to mimic or even exceed human capabilities. However, for applications involving continuous decision making in unknown environments, such as rovers for space exploration, robots, unmanned aerial vehicles, etc., explicit supervision and generation of labelled data set is extremely difficult and expensive. Reinforcement learning (RL) allows the agents to take decisions without any (human/external) supervision or training on labelled dataset. However, the conventional implementations of RL on advanced digital CPUs/GPUs incur a significantly large power dissipation owing to their inherent von-Neumann architecture. Although crossbar arrays of emerging non-volatile memories such as resistive (R)RAMs with their innate capability to perform energy-efficient in situ multiply-accumulate operation appear promising for Q-learning-based RL implementations, their limited endurance restricts their application in practical RL systems with overwhelming weight updates. To address this issue and realize the true potential of RRAM-based RL implementations, in this work, for the first time, we perform an algorithm-hardware co-design and propose a novel implementation of Monte Carlo (MC) RL algorithm on passive RRAM crossbar array. We analyse the performance of the proposed MC RL implementation on the classical cart-pole problem and demonstrate that it not only outperforms the prior digital and active 1-Transistor-1-RRAM (1T1R)-based implementations by more than five orders of magnitude in terms of area but is also robust against the spatial and temporal variations and endurance failure of RRAMs.

Adaptive Random Number Generator Based on RRAM Intrinsic Fluctuation for Reinforcement Learning

Implementation of A Chaos-Based ,High-Speed Truly Random Number Generator

Robustness of Neuromorphic Computing with RRAM-based Crossbars and Optical Neural Networks.

A High-performance and Calibration-free True Random Number Generator Based on the Resistance Perturbation in RRAM Array

A High-Speed True Random Number Generator Based on Unified Selector-RRAM

A Low Cost and High Reliability True Random Number Generator Based on Resistive Random Access Memory

A 3.3-Mbit/s True Random Number Generator Based on Resistive Random Access Memory

A RRAM-Based True Random Number Generator with 2T1R Architecture for Hardware Security Applications

Efficient Reinforcement Learning On Passive RRAM Crossbar Array

Demonstration of Generative Adversarial Network by Intrinsic Random Noises of Analog RRAM Devices

A 144-fJ/Bit Reliable and Compact TRNG Based on the Diffusive Resistance of 3-D Resistive Random Access Memory

A High-Speed and High-Reliability TRNG Based on Analog RRAM for IoT Security Application

Bayesian Neural Network Realization by Exploiting Inherent Stochastic Characteristics of Analog RRAM

Early-Stage Fluctuation in Low-Power Analog Resistive Memory: Impacts on Neural Network and Mitigation Approach

Impact of Circuit Limit and Device Noise on RRAM Based Conditional Generative Adversarial Network

A Seamless, Reconfigurable, and Highly Parallel In-Memory Stochastic Computing Approach With Resistive Random Access Memory Array

Intelligent Computing with RRAM

Optimized Programming Scheme Enabling Symmetric Conductance Modulation in HfO₂ Resistive Random-Access Memory (RRAM) for Neuromorphic Systems

A novel RRAM-based adaptive-threshold LIF neuron circuit for high recognition accuracy

A Physics-based Model of RRAM Probabilistic Switching for Generating Stable and Accurate Stochastic Bit-streams

An RRAM Retention Prediction Framework Using a Convolutional Neural Network Based on Relaxation Behavior