Abstract:Advances in deep learning have led to state-of-the-art performance across a multitude of speech recognition tasks. Nevertheless, the widespread deployment of deep neural networks for on-device speech recognition remains a challenge, particularly in edge scenarios where the memory and computing resources are highly constrained (e.g., low-power embedded devices) or where the memory and computing budget dedicated to speech recognition is low (e.g., mobile devices performing numerous tasks besides speech recognition). In this study, we introduce the concept of attention condensers for building low-footprint, highly-efficient deep neural networks for on-device speech recognition on the edge. An attention condenser is a self-attention mechanism that learns and produces a condensed embedding characterizing joint local and cross-channel activation relationships, and performs selective attention accordingly. To illustrate its efficacy, we introduce TinySpeech, low-precision deep neural networks comprising largely of attention condensers tailored for on-device speech recognition using a machine-driven design exploration strategy, with one tailored specifically with microcontroller operation constraints. Experimental results on the Google Speech Commands benchmark dataset for limited-vocabulary speech recognition showed that TinySpeech networks achieved significantly lower architectural complexity (as much as $507\times$ fewer parameters), lower computational complexity (as much as $48\times$ fewer multiply-add operations), and lower storage requirements (as much as $2028\times$ lower weight memory requirements) when compared to previous work. These results not only demonstrate the efficacy of attention condensers for building highly efficient networks for on-device speech recognition, but also illuminate its potential for accelerating deep learning on the edge and empowering TinyML applications.

Age Estimation from Speech Using Tuned CNN Model on Edge Devices

Convolutional Neural Network-Based Age Estimation Using B-Mode Ultrasound Tongue Image

Age and Gender Recognition Using a Convolutional Neural Network with a Specially Designed Multi-Attention Module through Speech Spectrograms

Phone duration modeling for speaker age estimation in children

Audio Tagging on an Embedded Hardware Platform

Performance Analysis of Deep Learning Model-Compression Techniques for Audio Classification on Edge Devices

Voice-Activated Emergency Response: Integrating Arduino Nicla Voice with MIT App Inventor for Elderly Care in Smart Homes

Efficient Real-Time Smart Keyword Spotting Using Spectrogram-Based Hybrid CNN-LSTM for Edge System

Real-Time Sound Source Localization for Low-Power IoT Devices Based on Multi-Stream CNN

A Real-Time and Privacy-Preserving Facial Expression Recognition System Using an AI-Powered Microcontroller

Edge Artificial Intelligence: Real-Time Noninvasive Technique for Vital Signs of Myocardial Infarction Recognition Using Jetson Nano

A Wearable Real-Time Character Recognition System Based on Edge Computing-Enabled Deep Learning for Air-Writing

TinySpeech: Attention Condensers for Deep Speech Recognition Neural Networks on Edge Devices

EdgeRNN: A Compact Speech Recognition Network With Spatio-Temporal Features for Edge Computing

Deep Learning Models in Speech Recognition: Measuring GPU Energy Consumption, Impact of Noise and Model Quantization for Edge Deployment

Age Estimation and Gender Recognition Technique Using Deep Learning

Optimizing Speech Recognition For The Edge

Artificial Intelligence Based Automatic Speech Emotion and Drunkenness Detection Using Convolutional Neural Network Voice Segment Dropout Optimizer

Analysis of facial emotion recognition rate for real-time application using NVIDIA Jetson Nano in deep learning models

Realtime Person Identification via Gait Analysis

Real Time Gender and Age Prediction using Deep Learning Techniques