Abstract:We consider feature learning for efficient keyword spotting that can be applied in severely under-resourced settings. The objective is to support humanitarian relief programmes by the United Nations in parts of Africa in which almost no language resources are available. For rapid development in such languages, we rely on a small, easily-compiled set of isolated keywords. These keyword templates are applied to a large corpus of in-domain but untranscribed speech using dynamic time warping (DTW). The resulting DTW alignment scores are used to train a convolutional neural network (CNN) which is orders of magnitude more computationally efficient and suitable for real-time application. We optimise this neural network keyword spotter by identifying robust acoustic features in this almost zero-resource setting. First, we incorporate information from well-resourced but unrelated languages using a multilingual bottleneck feature (BNF) extractor. Next, we consider features extracted from an autoencoder (AE) trained on in-domain but untranscribed data. Finally, we consider correspondence autoencoder (CAE) features which are fine-tuned on the small set of in-domain labelled data. Experiments in South African English and Luganda, a low-resource language, show that BNF and CAE features achieve a 5% relative performance improvement over baseline MFCCs. However, using BNFs as input to the CAE results in a more than 27% relative improvement over MFCCs in ROC area-under-the-curve (AUC) and more than twice as many top-10 retrievals. We show that, using these features, the CNN-DTW keyword spotter performs almost as well as the DTW keyword spotter while outperforming a baseline CNN trained only on the keyword templates. The CNN-DTW keyword spotter using BNF-derived CAE features represents an efficient approach with competitive performance suited to rapid deployment in a severely under-resourced scenario.

High-Performance Swahili Keyword Search with Very Limited Language Pack: the Thuee System for the Openkws15 Evaluation

The THUEE System for the Openkws14 Keyword Search Evaluation.

Subword scheme for keyword search

THUEE Language Modeling Method for the OpenKWS 2015 Evaluation

End-to-end keyword search system based on attention mechanism and energy scorer for low resource languages

Synth4Kws: Synthesized Speech for User Defined Keyword Spotting in Low Resource Environments

Joint Decoding of Tandem and Hybrid Systems for Improved Keyword Spotting on Low Resource Languages

Multilingual Query-by-Example Keyword Spotting with Metric Learning and Phoneme-to-Embedding Mapping

Feature learning for efficient ASR-free keyword spotting in low-resource languages

TDT-KWS: Fast And Accurate Keyword Spotting Using Token-and-duration Transducer

U2-KWS: Unified Two-pass Open-vocabulary Keyword Spotting with Keyword Bias

Investigation of Multilingual Deep Neural Networks for Spoken Term Detection.

The USYD-JD Speech Translation System for IWSLT 2021

Web-based keyword adapted Language Modeling for Keyword Spotting

GE2E-KWS: Generalized End-to-End Training and Evaluation for Zero-shot Keyword Spotting

Exploiting Noisy Web Data by OOV Ranking for Low-Resource Keyword Search.

Query-by-Example Keyword Spotting Using Spectral-Temporal Graph Attentive Pooling and Multi-Task Learning

THU-EE System Description for NIST LRE 2015

Pretraining End-to-End Keyword Search with Automatically Discovered Acoustic Units

Unified Keyword Spotting and Audio Tagging on Mobile Devices with Transformers

Keyword Spotting for Hearing Assistive Devices Robust to External Speakers