Abstract:Recent advances in Tiny Machine Learning (TinyML) empower low-footprint embedded devices for real-time on-device Machine Learning (ML). While many acknowledge the potential benefits of TinyML, its practical implementation presents unique challenges. This study aims to bridge the gap between prototyping single TinyML models and developing reliable TinyML systems in production: (1) Embedded devices operate in dynamically changing conditions. Existing TinyML solutions primarily focus on inference, with models trained offline on powerful machines and deployed as static objects. However, static models may underperform in the real world due to evolving input data distributions. We propose online learning to enable training on constrained devices, adapting local models toward the latest field conditions. (2) Nevertheless, current on-device learning methods struggle with heterogeneous deployment conditions and the scarcity of labeled data when applied across numerous devices. We introduce federated meta-learning incorporating online learning to enhance model generalization, facilitating rapid learning. This approach ensures optimal performance among distributed devices by knowledge sharing. (3) Moreover, TinyML’s pivotal advantage is widespread adoption. Embedded devices and TinyML models prioritize extreme efficiency, leading to diverse characteristics ranging from memory and sensors to model architectures. Given their diversity and non-standardized representations, managing these resources becomes challenging as TinyML systems scale up. We present semantic management for the joint management of models and devices at scale. We demonstrate our methods through a basic regression example and then assess them in three real-world TinyML applications: handwritten character image classification, keyword audio classification, and smart building presence detection. The results confirm the effectiveness of our approaches from various perspectives, such as accuracy improvement, resource savings, and engineering effort reduction.

A Model-Specific End-to-End Design Methodology for Resource-Constrained TinyML Hardware.

Toward Attention-based TinyML: A Heterogeneous Accelerated Architecture and Automated Deployment Flow

An Ultra-low Power TinyML System for Real-time Visual Processing at Edge

Supporting AI Engineering on the IoT Edge through Model-Driven TinyML

Tiny Machine Learning: Progress and Futures

A 28 Nm 0.25-0.61 Mw 31-60Fps Versatile SoC for Diverse Extreme Edge ML Workloads with Flexible Hetero-Fabric Dataflow Orchestration and Compute/Storage-Density-Adjustable CIM

Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review

On-device Online Learning and Semantic Management of TinyML Systems

TinyVers: A Tiny Versatile System-on-chip with State-Retentive eMRAM for ML Inference at the Extreme Edge

Low-Energy On-Device Personalization for MCUs

Introduction to the Special Issue on tinyML

A fine-grained mixed precision DNN accelerator using a two-stage big-little core RISC-V MCU.

A Heterogeneous TinyML SoC with Energy-Event-Performance-Aware Management and Compute-in-Memory Two-Stage Event-Driven Wakeup

iMCU: A 28-nm Digital In-Memory Computing-Based Microcontroller Unit for TinyML

A Machine Learning-oriented Survey on Tiny Machine Learning

TinyML: Tools, Applications, Challenges, and Future Research Directions

DTMM: Deploying TinyML Models on Extremely Weak IoT Devices with Pruning

U-TOE: Universal TinyML On-board Evaluation Toolkit for Low-Power IoT

Design of a Novel Neural Network Compression Method for Tiny Machine Learning