Abstract:Due to the separate memory and computation units in traditional von Neumann architecture, massive data transfer dominates the overall computing system's power and latency, known as the "Memory-Wall" issue. Especially with ever-increasing deep learning-based AI model size and computing complexity, it becomes the bottleneck for state-of-the-art AI computing systems. To address this challenge, in-memory computing (IMC)-based Neural Network accelerators have been widely investigated to support AI computing within memory. However, most of those works focus only on inference. The on-device training and continual learning have not been well explored yet. In this work, for the first time, we introduce on-device continual learning with STT-assisted-SOT (SAS) magnetoresistive random-access memory (MRAM)-based IMC system. On the hardware side, we have fabricated a STT-assisted-SOT MRAM (SAS-MRAM) device prototype with 4 magnetic tunnel junctions (MTJs, each at 100 nm x50 nm) sharing a common heavy metal layer, achieving significantly improved memory writing and area efficiency compared to traditional SOT-MRAM. Next, we designed fully digital IMC circuits with our SAS-MRAM to support both neural network inference and on-device learning. To enable efficient on-device continual learning for new task data, we present an 8-bit integer (INT8)-based continual learning algorithm that utilizes our SAS-MRAM IMC-supported bit-serial digital in-memory convolution operations to train a small parallel reprogramming network (Rep-Net) while freezing the major backbone model. Extensive studies have been presented based on our fabricated SAS-MRAM device prototype, cross-layer device-circuit benchmarking and simulation, as well as the on-device continual learning system evaluation.

Monolithic 3D Integration of Logic, Memory and Computing-In-Memory for One-Shot Learning

Monolithic Three-Dimensional Integration of RRAM-based Hybrid Memory Architecture for One-Shot Learning

A Hybrid Computing-In-Memory Architecture by Monolithic 3D Integration of BEOL CNT/IGZO-based CFET Logic and Analog RRAM

Monolithic 3D Integration of FeFET, Hybrid CMOS Logic and Analog RRAM Array for Energy-Efficient Reconfigurable Computing-In-Memory Architecture.

Monolithic 3-D Integration of Counteractive Coupling IGZO/CNT Hybrid 2T0C DRAM and Analog RRAM-Based Computing-In-Memory

Fully Integrated 3-D Stackable CNTFET/RRAM 1T1R Array As BEOL Buffer Macro for Monolithic 3-D Integration with Analog RRAM-Based Computing-in-Memory

3D Stackable CNTFET/RRAM 1T1R Array with CNT CMOS Peripheral Circuits as BEOL Buffer Macro for Monolithic 3D Integration with Analog RRAM-based Computing-In-Memory

Monolithic 3D Integration of Dendritic Neural Network with Memristive Synapse, Dendrite and Soma on Si CMOS

Ferroelectric FET Based In-Memory Computing for Few-Shot Learning.

Counteractive Coupling IGZO/CNT Hybrid 2T0C DRAM Accelerating RRAM-based Computing-In-Memory Via Monolithic 3D Integration for Edge AI

Monolithic 3D Integration of Analog RRAM‐Based Computing‐in‐Memory and Sensor for Energy‐Efficient Near‐Sensor Computing (Adv. Mater. 22/2024)

SAPIENS: A 64-kb RRAM-Based Non-Volatile Associative Memory for One-Shot Learning and Inference at the Edge

Reconfigurable 3D Stacked Oxide Semiconductor Devices for Hybrid In-Memory Computing

A Monolithic-3D SRAM Design with Enhanced Robustness and In-Memory Computation Support

System-Technology Co-Optimization for 3D Monolithic Memory-Centric Computing

Monolithic 3D integration of 2D transistors and vertical RRAMs in 1T–4R structure for high-density memory

Monolithic 3D integration of logic and memory: Carbon nanotube FETs, resistive RAM, and silicon FETs

On-Device Continual Learning with STT-Assisted-SOT MRAM Based In-Memory Computing

1-Mbit 3-D DRAM Using a Monolithically Stacked Structure of a Si CMOS and Heterogeneous IGZO FETs

33.2 A Fully Integrated Analog ReRAM Based 78.4TOPS/W Compute-In-Memory Chip with Fully Parallel MAC Computing.

Ferroelectric Ternary Content-Addressable Memory for One-Shot Learning