Abstract:Deep neural networks (DNNs) have recently achieved impressive success across a wide range of real-world vision and language processing tasks, spanning from image classification to many other downstream vision tasks, such as object detection, tracking, and segmentation. However, previous well-established DNNs, despite being able to maintain superior accuracy, have also been evolving to be deeper and wider and thus inevitably necessitate prohibitive computational resources for both training and inference. This trend further enlarges the computational gap between computation-intensive DNNs and resource-constrained embedded computing systems, making it challenging to deploy powerful DNNs upon real-world embedded computing systems towards ubiquitous embedded intelligence. To alleviate the above computational gap and enable ubiquitous embedded intelligence, we, in this survey, focus on discussing recent efficient deep learning infrastructures for embedded computing systems, spanning from training to inference, from manual to automated, from convolutional neural networks to transformers, from transformers to vision transformers, from vision models to large language models, from software to hardware, and from algorithms to applications. Specifically, we discuss recent efficient deep learning infrastructures for embedded computing systems from the lens of (1) efficient manual network design for embedded computing systems, (2) efficient automated network design for embedded computing systems, (3) efficient network compression for embedded computing systems, (4) efficient on-device learning for embedded computing systems, (5) efficient large language models for embedded computing systems, (6) efficient deep learning software and hardware for embedded computing systems, and (7) efficient intelligent applications for embedded computing systems.

Efficient DNN Algorithm Design and Hardware Acceleration for Low-Level Vision

Survey on Energy-Efficient Deep Neural Networks for Computer Vision

DaDianNao: A Machine-Learning Supercomputer

Efficient Hardware Optimization Strategies For Deep Neural Networks Acceleration Chip

Hardware Accelerator Design for Sparse DNN Inference and Training: A Tutorial

A Survey on Efficient Convolutional Neural Networks and Hardware Acceleration

Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework

Hardware Implementation of Energy Efficient Deep Learning Neural Network Based on Nanoscale Flash Computing Array

3U-EdgeAI: Ultra-Low Memory Training, Ultra-Low BitwidthQuantization, and Ultra-Low Latency Acceleration

Invited: Algorithm-Software-Hardware Co-Design for Deep Learning Acceleration

Design and Implementation of Deep Neural Network for Edge Computing

Recent Advances in Efficient Computation of Deep Convolutional Neural Networks

VisualNet: an End-to-End Human Visual System Inspired Framework to Reduce Inference Latency of Deep Neural Networks

Design Automation for Efficient Deep Learning Computing

Toward Efficient and Adaptive Design of Video Detection System with Deep Neural Networks

Algorithm and hardware codesign of sparse binary network on-chip

Learning on Hardware: A Tutorial on Neural Network Accelerators and Co-Processors

Efficient Deep Learning Infrastructures for Embedded Computing Systems: A Comprehensive Survey and Future Envision

Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications

An Efficient Hardware Architecture for DNN Training by Exploiting Triple Sparsity

Deep Learning Hardware Acceleration Based on General Vector DSP