Abstract:Keyword spotting (KWS) systems are used for human-machine communications in various applications. In many cases, KWS involves a combination of wake-up-word (WUW) recognition for device activation and voice command classification tasks. These tasks present a challenge for embedded systems due to the complexity of deep learning algorithms and the need for optimized networks for each application. In this paper, we propose a depthwise separable binarized/ternarized neural network (DS-BTNN) hardware accelerator capable of performing both WUW recognition and command classification on a single device. The design achieves significant area efficiency by redundantly utilizing bitwise operators in the computation of the binarized neural network (BNN) and ternary neural network (TNN). In a complementary metal-oxide semiconductor (CMOS) 40 nm process environment, the DS-BTNN accelerator demonstrated significant efficiency. Compared with a design approach where BNN and TNN were independently developed and subsequently integrated as two separate modules into the system, our method achieved a 49.3% area reduction while yielding an area of 0.558 mm2. The designed KWS system, which was implemented on a Xilinx UltraScale+ ZCU104 field-programmable gate array (FPGA) board, receives real-time data from the microphone, preprocesses them into a mel spectrogram, and uses this as input to the classifier. Depending on the order, the network operates as a BNN or a TNN for WUW recognition and command classification, respectively. Operating at 170 MHz, our system achieved 97.1% accuracy in BNN-based WUW recognition and 90.5% in TNN-based command classification.

An Efficient Keywords Spotting System with Speaker Verification Based on Binary Neural Networks

An Energy-Efficient Binarized Neural Network Using Analog-Intensive Feature Extraction for Keyword and Speaker Verification Wakeup.

Focal Loss And Double-Edge-Triggered Detector For Robust Small-Footprint Keyword Spotting

End-to-End Speech Keyword Spotting System

Low Bits: Binary Neural Network for Vad and Wakeup

Residual Spiking Neural Network on a Programmable Neuromorphic Hardware for Speech Keyword Spotting

Wake-up-word Spotting Using End-to-end Deep Neural Network System.

FPGA Implementation of Keyword Spotting System Using Depthwise Separable Binarized and Ternarized Neural Networks

An Always-on Ultra-Low Power Speaker Verification Accelerator Based on Binary Weighted Neural Network with System Co-optimization

Hello Edge: Keyword Spotting on Microcontrollers

A Depthwise Separable Convolution Neural Network for Small-footprint Keyword Spotting Using Approximate MAC Unit and Streaming Convolution Reuse

Background Noise Adaptive Energy-Efficient Keywords Recognition Processor with Reusable DNN and Reconfigurable Architecture

Effective Combination of DenseNet and BiLSTM for Keyword Spotting.

A 3.8-Μw 10-Keyword Noise-Robust Keyword Spotting Processor Using Symmetric Compressed Ternary-Weight Neural Networks

Model compression applied to small-footprint keyword spotting

A Study of Designing Compact Audio-Visual Wake Word Spotting System Based on Iterative Fine-Tuning in Neural Network Pruning

Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting

A 510-nW Wake-Up Keyword-Spotting Chip Using Serial-FFT-Based MFCC and Binarized Depthwise Separable CNN in 28-nm CMOS

An Ultra-low Power Keyword-Spotting Accelerator Using Circuit-Architecture-System Co-design and Self-adaptive Approximate Computing Based BWN

Monophone-Based Background Modeling for Two-Stage On-Device Wake Word Detection

Improved Keyword Spotting System by Optimizing Posterior Confidence Measure Vector Using Feed-Forward Neural Network.