Abstract:Due to the separate memory and computation units in traditional von Neumann architecture, massive data transfer dominates the overall computing system's power and latency, known as the "Memory-Wall" issue. Especially with ever-increasing deep learning-based AI model size and computing complexity, it becomes the bottleneck for state-of-the-art AI computing systems. To address this challenge, in-memory computing (IMC)-based Neural Network accelerators have been widely investigated to support AI computing within memory. However, most of those works focus only on inference. The on-device training and continual learning have not been well explored yet. In this work, for the first time, we introduce on-device continual learning with STT-assisted-SOT (SAS) magnetoresistive random-access memory (MRAM)-based IMC system. On the hardware side, we have fabricated a STT-assisted-SOT MRAM (SAS-MRAM) device prototype with 4 magnetic tunnel junctions (MTJs, each at 100 nm x50 nm) sharing a common heavy metal layer, achieving significantly improved memory writing and area efficiency compared to traditional SOT-MRAM. Next, we designed fully digital IMC circuits with our SAS-MRAM to support both neural network inference and on-device learning. To enable efficient on-device continual learning for new task data, we present an 8-bit integer (INT8)-based continual learning algorithm that utilizes our SAS-MRAM IMC-supported bit-serial digital in-memory convolution operations to train a small parallel reprogramming network (Rep-Net) while freezing the major backbone model. Extensive studies have been presented based on our fabricated SAS-MRAM device prototype, cross-layer device-circuit benchmarking and simulation, as well as the on-device continual learning system evaluation.

MnM: A Fast and Efficient Min/Max Searching in MRAM

Max-PIM: Fast and Efficient Max/Min Searching in DRAM

Ultrafast Self-Rectifying Memristors for Advanced In-Memory Computing

Modeling and Benchmarking Computing-in-Memory for Design Space Exploration.

A 6T-3M SOT-MRAM for In-Memory Computing with Reconfigurable Arithmetic Operations.

A6T-3MSOT-MRAMforin-memorycomputingwith reconfigurable arithmetic operations

High performance and energy-efficient in-memory computing architecture based on SOT-MRAM

Low Power In-Memory Computing Based On Dual-Mode Sot-Mram

Resistance-Sum Architecture for Voltage-Controlled SOT-MRAM based Computing-in-Memory With Hybrid References

Exploring a SOT-MRAM Based In-Memory Computing for Data Processing.

In-Memory Computing with Spintronic Devices

Implementation of Highly Reliable and Energy‐Efficient Nonvolatile In‐Memory Computing Using Multistate Domain Wall Spin–Orbit Torque Device

Proposal of Analog In-Memory Computing with Magnified Tunnel Magnetoresistance Ratio and Universal STT-MRAM Cell

A survey of in-spin transfer torque MRAM computing

Parallel-Prefix Adder in Spin-Orbit Torque Magnetic RAM for High Bit-Width Non-Volatile Computation

MRIMA: an MRAM-Based In-Memory Accelerator

Leveraging Spintronic Devices for Ultra-Low Power In-Memory Computing: Logic and Neural Network.

All-Electrical Control of Compact SOT-MRAM: Toward Highly Efficient and Reliable Non-Volatile In-Memory Computing

On-Device Continual Learning with STT-Assisted-SOT MRAM Based In-Memory Computing

Data Matching Engine for Efficient Big Data Searching Systems Using a RRAM Based Novel Computing Paradigm

NAND-SPIN-based processing-in-MRAM architecture for convolutional neural network acceleration