Abstract:Tiny machine learning (TinyML) envisions executing a deep neural network (DNN)-based inference on an edge device for improving battery life, latency, security, and privacy. Toward this vision, recent microcontroller units (MCUs) integrate in-memory computing (IMC) hardware to leverage its high energy efficiency and throughput in vector–matrix multiplication (VMM). However, those existing works require large IMC hardware, severely increasing the area overhead. In addition, most existing works use analog–mixed-signal (AMS) IMC hardware, exhibiting limited robustness over process, voltage, and temperature (PVT) variations. Finally, none can support a practical software development framework such as TensorFlow Lite for Microcontrollers (TFLite-micro). Due to these limitations, those MCUs did not present the performance for the standard benchmark MLPerf-Tiny, which makes it difficult to evaluate them against the state-of-the-art neural (not necessarily IMC-based) MCUs. In this article, we design a new IMC-based MCU, titled iMCU, for TinyML to address those challenges. In the design process, we: 1) define the optimal set of acceleration targets and 2) devise an area-efficient computation flow that requires the least amount of IMC hardware yet still provides a significant acceleration. In addition, we develop: 1) state-of-the-art digital IMC macros and 2) create the accelerator based on the macros, which can support the proposed computation flow in a fully pipelined manner. Combining those innovations, we prototyped the iMCU in a 28-nm CMOS. The measurement results show that the iMCU significantly outperforms the prior IMC-based MCUs in compute density, energy efficiency, and SRAM density (total SRAM size/total SRAM area). It also achieves a compact footprint of 2.73 mm $^2$ .

An End-to-End in Memory Computing System Based on A 40nm Eflash-Based IMC SoC: Circuits, Toolchains, and Systems Co-Design Framework

A Robust 8-Bit Non-Volatile Computing-in-Memory Core for Low-Power Parallel MAC Operations.

Neural Network Acceleration and Voice Recognition with a Flash-based In-Memory Computing SoC

Computing-in-memory Circuits and Cross-Layer Integrated Design and Optimization: from SRAM to FeFET

In-Memory Computing Integrated Structure Circuit Based on Nonvolatile Flash Memory Unit

An in-memory computing architecture based on a duplex 2D material structure for in-situ machine learning

A 64-core mixed-signal in-memory compute chip based on phase-change memory for deep neural network inference

A Heterogeneous Microprocessor Based on All-Digital Compute-in-Memory for End-to-End AIoT Inference

Design and Implementation of a Charge-Sharing In-Memory-computing Macro with Sparse Feature for Quantized Neural Network

A Fully Digital SRAM-Based Four-Layer In-Memory Computing Unit Achieving Multiplication Operations and Results Store

MLFlash-CIM: Embedded Multi-Level NOR-Flash Cell based Computing in Memory Architecture for Edge AI Devices

A Heterogeneous Microprocessor for Intermittent AI Inference Using Nonvolatile-SRAM-based Compute-In-Memory

In-Memory Computing: The Next-Generation AI Computing Paradigm

A Fully Integrated System‐on‐Chip Design with Scalable Resistive Random‐Access Memory Tile Design for Analog in‐Memory Computing

A Heterogeneous In-Memory Computing Cluster For Flexible End-to-End Inference of Real-World Deep Neural Networks

Device and Circuit Architectures for In‐Memory Computing

Hadamard Product-Based In-Memory Computing Design for Floating Point Neural Network Training

Design memristor-based computing-in-memory for AI accelerators considering the interplay between devices, circuits, and system

IMCE: Energy-efficient Bit-Wise In-Memory Convolution Engine for Deep Neural Network

iMCU: A 28-nm Digital In-Memory Computing-Based Microcontroller Unit for TinyML

33.2 A Fully Integrated Analog ReRAM Based 78.4TOPS/W Compute-In-Memory Chip with Fully Parallel MAC Computing.