Abstract:Tiny machine learning (TinyML) envisions executing a deep neural network (DNN)-based inference on an edge device for improving battery life, latency, security, and privacy. Toward this vision, recent microcontroller units (MCUs) integrate in-memory computing (IMC) hardware to leverage its high energy efficiency and throughput in vector–matrix multiplication (VMM). However, those existing works require large IMC hardware, severely increasing the area overhead. In addition, most existing works use analog–mixed-signal (AMS) IMC hardware, exhibiting limited robustness over process, voltage, and temperature (PVT) variations. Finally, none can support a practical software development framework such as TensorFlow Lite for Microcontrollers (TFLite-micro). Due to these limitations, those MCUs did not present the performance for the standard benchmark MLPerf-Tiny, which makes it difficult to evaluate them against the state-of-the-art neural (not necessarily IMC-based) MCUs. In this article, we design a new IMC-based MCU, titled iMCU, for TinyML to address those challenges. In the design process, we: 1) define the optimal set of acceleration targets and 2) devise an area-efficient computation flow that requires the least amount of IMC hardware yet still provides a significant acceleration. In addition, we develop: 1) state-of-the-art digital IMC macros and 2) create the accelerator based on the macros, which can support the proposed computation flow in a fully pipelined manner. Combining those innovations, we prototyped the iMCU in a 28-nm CMOS. The measurement results show that the iMCU significantly outperforms the prior IMC-based MCUs in compute density, energy efficiency, and SRAM density (total SRAM size/total SRAM area). It also achieves a compact footprint of 2.73 mm $^2$ .

A Heterogeneous TinyML SoC with Energy-Event-Performance-Aware Management and Compute-in-Memory Two-Stage Event-Driven Wakeup

A Low-Power In-Memory Multiplication and Accumulation Array with Modified Radix-4 Input and Canonical Signed Digit Weights

TinyVers: A Tiny Versatile System-on-chip with State-Retentive eMRAM for ML Inference at the Extreme Edge

A 28 Nm 0.25-0.61 Mw 31-60Fps Versatile SoC for Diverse Extreme Edge ML Workloads with Flexible Hetero-Fabric Dataflow Orchestration and Compute/Storage-Density-Adjustable CIM

OPCIO: Optimizing Power Consumption for Embedded Devices via GPIO Configuration

A 148-Nw Reconfigurable Event-Driven Intelligent Wake-Up System for AIoT Nodes Using an Asynchronous Pulse-Based Feature Extractor and a Convolutional Neural Network

12.1 A 148nw General-Purpose Event-Driven Intelligent Wake-Up Chip for AIoT Devices Using Asynchronous Spike-Based Feature Extractor and Convolutional Neural Network

20.2 A 57nw Software-Defined Always-On Wake-Up Chip for IoT Devices with Asynchronous Pipelined Event-Driven Architecture and Time-Shielding Level-Crossing ADC

An Ultra-low Power TinyML System for Real-time Visual Processing at Edge

A Software-Defined Always-On System With 57-75-nW Wake-Up Function Using Asynchronous Clock-Free Pipelined Event-Driven Architecture and Time-Shielding Level-Crossing ADC

A Sub-$\mu$W Energy-Performance-Aware IoT SoC With a Triple-Mode Power Management Unit for System Performance Scaling, Fast DVFS, and Energy Minimization

Decoupled Access-Execute enabled DVFS for tinyML deployments on STM32 microcontrollers

Low-Energy On-Device Personalization for MCUs

A Model-Specific End-to-End Design Methodology for Resource-Constrained TinyML Hardware.

A Heterogeneous Microprocessor Based on All-Digital Compute-in-Memory for End-to-End AIoT Inference

iMCU: A 28-nm Digital In-Memory Computing-Based Microcontroller Unit for TinyML

A Heterogeneous Microprocessor for Intermittent AI Inference Using Nonvolatile-SRAM-based Compute-In-Memory

Toward Attention-based TinyML: A Heterogeneous Accelerated Architecture and Automated Deployment Flow

Battery Management Technique to Reduce Standby Energy Consumption in Ultra-Low Power IoT and Sensory Applications

A Sub-100nA Ultra-low Leakage MCU Embedding Always-on Domain Hybrid Tunnel FET-CMOS on 300mm Foundry Platform

Deep Reinforcement Learning-Based Power Management for Chiplet-Based Multicore Systems