An Approach of Keyword Spotting Based on HMM

BF Yan,R Guo,XY Zhu,B Zhang

DOI: https://doi.org/10.1109/wcica.2000.862561

2000-01-01

Abstract:Keyword spotting is one of important topics in speech recognition. The paper presents a model of keyword spotting and the training method of relevant keyword models and anti-words models, then we give a strategy of keyword spotting. Experiment indicates the strategy is effective

What problem does this paper attempt to address?

A Keyword Spotting Method

Guo Rui,Zhu Xiaoyan

2000-01-01

Abstract:In this paper a new approach of keyboard spotting is proposed and its basic theory and algorithms are described. By applying this method a speaker independent and vocabulary independent Chinese keyword spotter based on hidden Markov model(HMM) is implemented. The filler models used in the keyword detection are trained from Chinese syllables through clustering technique, and then antikeyword models \5 \ are introduced and applied in the keyword verification. To evaluate the performance, a lot of experiments are carried out, so statistical results are presented and some important problems in keyword spotting are investigated.
A New Keyword Spotting Approach for Spontaneous Mandarin Speech

Pengyuan Zhang,Jiang Han,Jian Shao,Yonghong Yan

DOI: https://doi.org/10.1109/icosp.2006.345535

2006-01-01

Abstract:For many practical applications of keyword spotting, input signal is a spontaneous conversation. Generally speaking, keyword spotting system will degrade significantly because of mismatch between acoustic model and speech. To solve this problem, this paper presents a two-pass based keyword spotting strategy. Different from one-pass based system, decoding process is done in the whole acoustic space, not especially for keywords space. 1-best phoneme sequence is achieved during the decoding process firstly. Then keywords are extracted by a keyword list and confusion matrix in the second pass. At last, the phoneme lattice based confidence measure is adopted to reject false alarms. Experiments show that two-pass based strategy performances a significant improvement on the spontaneous telephone conversations
Experimental Investigation into Alignment-based Acoustic Confidence Measures in Keyword Verification for Mandarin Speech

Yiyan Liu,Yingchun Yang,Zhenyu Shan

2006-01-01

Abstract:This paper introduces the methods to verify keyword hypothesis using alignment information, which can be easily implemented and integrated into the keyword spotting system in a straightforward way. Acquiring alignment information goes without any extra model training, and has no evident total processing time impact on original keyword spotting system. Different alignment-based confidence measures are evaluate using ROC curves. In addition, we introduce a novel evaluation metric, Confidence Error Cost Function (CECF), which improves upon Confidence Error Rate (CER). It is more general than CER since it takes different cost weights of FA and FR into consideration. We present the result of experiments aiming at assessing the quality and the limitations of different confidence measures and find out that the combination of minimum alignment cost and state duration normalized posterior probability gives the best performance.
Keyword-specific normalization based keyword spotting for spontaneous speech

Li W,Liao Q.

DOI: https://doi.org/10.1109/ISCSLP.2012.6423490

2012-01-01

Abstract:This paper presents a novel architecture for keyword spotting in spontaneous speech, in which keyword model is trained from a small number of acoustic examples provided by a user. The word-spotting architecture relies on scoring patch feature vector sequences extracted by using sliding windows, and performing keyword-specific normalization and threshold setting. Dynamic time warping (DTW) based template matching and Gaussian Mixture Models (GMM) are proposed to model the keyword, and another GMM is proposed to model the non-keywords. Our keyword spotting experiments demonstrate the effectiveness of the proposed methods. More specifically, the proposed GMM log-likelihood ratio based method achieves about 17% absolute improvement in terms of recall rates compared to the baseline system.
A VOCABULARY-INDEPENDENT KEYWORD SPOTTER FOR SPONTANEOUS CHINESE SPEECH

ZHENG Fang,XU Mingxing,MOU Xiaolong,WU Jian,WU Wenhu,FANG Ditang

2013-01-01

Abstract:HarkMan keyword-spotter was designed so that it can be used in a real-world environment to automatically spot the given words of a vocabulary-independent (VIND) task in unconstrained Chinese telephone speech. In this spotter, the speaking manner and the number of keywords are not limited. This paper focuses on a novel technique that addresses acoustic modeling, keyword-spotting network, search strategies, robustness, and rejection adopted in HarkMan. The underlying technologies used in HarkMan given in this paper are not only for keyword spotting but also for continuous speech recognition, which had been proved very efficient. It achieved the figure-of-merit (FOM) value over 90%.
Keyword Spotting Based on Hypothesis Boundary Realignment and State-Level Confidence Weighting

Hong Liu,Yuezhao Chen,Runwei Ding,Cheng Pang

DOI: https://doi.org/10.1145/3207677.3277918

2018-01-01

Abstract:Keyword(1) spotting (KWS) deals with the identification of keywords in speech utterances. A two-stage approach is often used for the flexibility and high efficiency. The two stages are keyword hypotheses detection stage and hit or false-alarm verification stage in sequence. How to reduce the false-alarms is a key and difficult problem in the verification stage, which is formatted as the confidence measure (CM) problem. In this paper, a novel keyword-filler hidden Markov model (HMM) based method is proposed based on two improved approaches. On one hand, for more effective confidence measure, a hypothesis boundary realignment method is used to gain more precise hypothesized segments for possible keyword. Then an overlap ratio criterion is defined to evaluate this process. On the other hand, a state-level confidence weighting method is proposed to improve the posterior probability based CM. Experiments show that either improvement is effective, and the proposed method based on the two processes gives the best performance.
HarkMan—A Vocabulary-Independent Keyword Spotter for Spontaneous Chinese Speech

Zheng Fang,Xu Mingxing,Mou Xiaolong,Wu Jian,Wu Wenhu,Fang Ditang

DOI: https://doi.org/10.1007/bf02952483

IF: 1.871

1999-01-01

Journal of Computer Science and Technology

Abstract:In this paper, a novel technique adopted in HarkMan is introduced. HarkMan is a keyword-spotter designed to automatically spot the given words of a vocabulary-independent task in unconstrained Chinese telephone speech. The speaking manner and the number of keywords are not limited. This paper focuses on the novel technique which addresses acoustic modeling, keyword spotting network, search strategies, robustness, and rejection. The underlying technologies used in HarkMan given in this paper are useful not only for keyword spotting but also for continuous speech recognition. The system has achieved a figure-of-merit value over 90%.
Keyword Spotting Based on Phoneme Confusion Matrix

Pengyuan Zhang,Jian Shao,Jiang Han,Zhaojie Liu,Yonghong Yan

2006-01-01

Abstract:For many practical applications of keyword spotting, input signal is a spontaneous conversation while the acoustic model was trained with read speech because of data availability. Generally speaking, keyword spotting system will degrade significantly because of mismatch between acoustic model and spontaneous speech. To solve this problem, this paper presents a two-pass keyword spotting strategy. In order to improve the retrieval performance, an improved phoneme confusion matrix is adopted. It will give more freedom in the representation so as to alleviate the effect of mismatched training condition and of phoneme misrecognition. Furthermore, a hybrid confidence measure is applied to reject false alarms. Experiments show that the proposed algorithms significantly reduced equal error rate (EER) on the telephone conversational task.
A Two-Step Keyword Spotting Method Using Fuzzy Search Algorithm

Zhang Pengyuan,Shao Jian,Yan Yonghong

IF: 1.019

2008-01-01

Chinese Journal of Electronics

Abstract:For many practical applications, input signal is a spontaneous conversation while the acoustic model was trained with read speech because of data availability. The performance of Keyword spotting system will degrade significantly. To solve this problem, we presented a two-step keyword spotting strategy. A fuzzy search algorithm was adopted to improve the retrieval performance. We generated confusion matrix from confusion network. It gives more freedom in the representation so as to alleviate the effect phoneme misrecognition. Furthermore, a hybrid confidence measure was applied to reject false accepts. Experiments show that the proposed algorithms significantly reduce Equal error rate (EER) on the telephone conversational task.
Keyword Spotting Based on Syllable Confusion Network.

Pengyuan Zhang,Jian Shao,Qingwei Zhao,Yonghong Yan

DOI: https://doi.org/10.1109/icnc.2007.457

2007-01-01

Abstract:Keyword spotting becomes a very important branch of speech recognition. But the acoustic mismatch between training and testing environments often causes a severe degradation in the recognition performance. This paper presents an improved keyword spotting strategy. A fuzzy search algorithm is proposed to extract keyword hypotheses from a syllable confusion network (SCN). SCN is linear and naturally suitable for indexing. To accelerate search process, SCN are pruned to feasible sizes. As a post-processing method, minimum classification error (MCE) optimized confidence measure is adopted to reject false accepts. On Mandarin conversational telephone speech (CTS), the proposed algorithms reduce the equal error rate (EER) by 7.2% relative.
A Multi-oriented Chinese Keyword Spotter Guided by Text Line Detection

Pei Xu,Shan Huang,Hongzhen Wang,Hao Song,Shen Huang,Qi Ju

DOI: https://doi.org/10.48550/arXiv.2001.00722

2020-01-06

Abstract:Chinese keyword spotting is a challenging task as there is no visual blank for Chinese words. Different from English words which are split naturally by visual blanks, Chinese words are generally split only by semantic information. In this paper, we propose a new Chinese keyword spotter for natural images, which is inspired by Mask R-CNN. We propose to predict the keyword masks guided by text line detection. Firstly, proposals of text lines are generated by Faster R-CNN;Then, text line masks and keyword masks are predicted by segmentation in the proposals. In this way, the text lines and keywords are predicted in parallel. We create two Chinese keyword datasets based on RCTW-17 and ICPR MTWI2018 to verify the effectiveness of our method.

Computer Vision and Pattern Recognition
An utterance verification algorithm in keyword spotting system

Haisheng Dai,Xiaoyan Zhu,Yupin Luo,Shiyuan Yang

DOI: https://doi.org/10.1007/11492542_68

2005-01-01

Abstract:Speech Recognition and Verification are important components of a Keyword Spotting System whose performance is determined by both of them. In this paper, a new model-distance based utterance verification algorithm is proposed to improve the performance of the keyword spotting system. Furthermore, a substitution-error recovery module is built for improving of the system, which enhances the detection rate with the false-alarm rate remaining almost the same. Experiment shows that with the novel utterance verification method and new added substitution-error recovery module, the system performance is improved greatly.
A New Framework for Large Vocabulary Keyword Spotting Using Two-Pass Confidence Measure

Yingna Chen,Tao Hou,Sha Meng,Shan Zhong,Jia Liu

DOI: https://doi.org/10.1109/cesa.2006.4281625

2006-01-01

Abstract:In this paper, a new framework for large vocabulary keyword spotting is proposed, which involves three phases. In the first phase, N-best sub-word lattice is generated by hidden Markov model (HMM). Keyword candidates are hypothesized by dynamic keyword matching during the second phase. In the last phase, two-pass confidence measure, which provides complementary information, is used for keyword verification. Experimental results show that, with the use of these improvements, the keyword spotting system proves to be more accurate and robust without much computation cost.
Keyword spotting based on mixed grammar model

Yining Chen,Jing Liu,Lin Zhong,Jia Liu,RunSheng Liu

DOI: https://doi.org/10.1109/isimp.2001.925424

2001-01-01

Abstract:We present a novel keyword spotting method based on the mixed grammar model. By merging the filler model and the finite state grammar, two conventional technologies of keyword spotting, the mixed grammar model incorporates both a priori knowledge and the capability of covering all possible sentential forms in real speech, thus makes up for the weaknesses of both parental technologies. Experimental results show that the mixed grammar model excels the filler model in overall performance and the finite state grammar in robustness. The expansibility of the mixed grammar model is shown in its capacity of easy incorporation of further improvement of both the filler model and finite state grammar
LEXICAL ACCESS-BASED CONFIDENCE MEASURE FOR A SPANISH KEYWORD SPOTTING SYSTEM

Javier Tejedor,Dong Wang,Simon King,Jose Colas

2007-01-01

Abstract:Keyword spotting deals with the search of a reduced set of keywords in audio content. Phone Lattice-based approaches are very fast but achieve poor results. HMM-based keyword spotting systems deal with filler models to absorb the Out-of-vocabulary (OOV) words and achieve best results although they are slower. We propose a technique which combines them in order to perform a confidence measure to reduce the false alarm rate achieved in the HMM-based keyword spotting module over a Spanish Keyword Spotting system. Different filler models are investigated within the experiments.
A Two-Step Keyword Spotting Method Based on Context-Dependent a Posteriori Probability

TF Zheng,J Li,ZJ Song,MX Xu

DOI: https://doi.org/10.1109/chinsl.2004.1409641

2004-01-01

Abstract:Keyword weighting plays an important role in traditional keyword spotting (KWS) systems: it helps detect keyword candidates in an utterance so that they will not be missed. However, if the keywords are over-weighted, there will be a high number of false alarms, which will slow down the system and might introduce rejection errors; on the other hand, if the keywords are insufficiently weighted, the detection rate is not guaranteed. It is difficult to make a compromise with regard to keyword weighting. A two-step KWS method based on context-dependent a posteriori probability (CDAPP) is proposed in this paper as a way to solve this problem. The first step adopts a continuous speech recognition method, to generate a sequence of acoustic symbols for the second step, which performs a fuzzy keyword search. Preliminary experiments show that the proposed strategy is a promising one that needs additional investigation.
MM-KWS: Multi-modal Prompts for Multilingual User-defined Keyword Spotting

Zhiqi Ai,Zhiyong Chen,Shugong Xu

2024-06-11

Abstract:In this paper, we propose MM-KWS, a novel approach to user-defined keyword spotting leveraging multi-modal enrollments of text and speech templates. Unlike previous methods that focus solely on either text or speech features, MM-KWS extracts phoneme, text, and speech embeddings from both modalities. These embeddings are then compared with the query speech embedding to detect the target keywords. To ensure the applicability of MM-KWS across diverse languages, we utilize a feature extractor incorporating several multilingual pre-trained models. Subsequently, we validate its effectiveness on Mandarin and English tasks. In addition, we have integrated advanced data augmentation tools for hard case mining to enhance MM-KWS in distinguishing confusable words. Experimental results on the LibriPhrase and WenetPhrase datasets demonstrate that MM-KWS outperforms prior methods significantly.

Audio and Speech Processing,Computation and Language,Sound
Word Spotting Based on a Posterior Measure of Keyword Confidence

Hao Jie,Li Xing

DOI: https://doi.org/10.1007/bf02943289

2002-01-01

Abstract:In this paper, an approach of keyword confidence estimation is developed that well combines acoustic layer scores and syllable-based statistical language model (LM) scores. An a posteriori (AP) confidence measure and its forward-backward calculating algorithm are deduced. A zero false alarm (ZFA) assumption is proposed for evaluating relative confidence measures by word spotting task. In a word spotting experiment with a vocabulary of 240 keywords, the keyword accuracy under the AP measure is above 94%, which well approaches its theoretical upper limit. In addition, a syllable lattice Hidden Markov Model (SLHMM) is formulated and a unified view of confidence estimation, word spotting, optimal path search, and N-best syllable re-scoring is presented. The proposed AP measure can be easily applied to various speech recognition systems as well.
Keyword spotting -- Detecting commands in speech using deep learning

Sumedha Rai,Tong Li,Bella Lyu

2023-12-10

Abstract:Speech recognition has become an important task in the development of machine learning and artificial intelligence. In this study, we explore the important task of keyword spotting using speech recognition machine learning and deep learning techniques. We implement feature engineering by converting raw waveforms to Mel Frequency Cepstral Coefficients (MFCCs), which we use as inputs to our models. We experiment with several different algorithms such as Hidden Markov Model with Gaussian Mixture, Convolutional Neural Networks and variants of Recurrent Neural Networks including Long Short-Term Memory and the Attention mechanism. In our experiments, RNN with BiLSTM and Attention achieves the best performance with an accuracy of 93.9 %

Sound,Artificial Intelligence,Computation and Language,Human-Computer Interaction,Audio and Speech Processing
Metric Learning for Keyword Spotting

Jaesung Huh,Minjae Lee,Heesoo Heo,Seongkyu Mun,Joon Son Chung

DOI: https://doi.org/10.48550/arXiv.2005.08776

2020-05-18

Abstract:The goal of this work is to train effective representations for keyword spotting via metric learning. Most existing works address keyword spotting as a closed-set classification problem, where both target and non-target keywords are predefined. Therefore, prevailing classifier-based keyword spotting systems perform poorly on non-target sounds which are unseen during the training stage, causing high false alarm rates in real-world scenarios. In reality, keyword spotting is a detection problem where predefined target keywords are detected from a variety of unknown sounds. This shares many similarities to metric learning problems in that the unseen and unknown non-target sounds must be clearly differentiated from the target keywords. However, a key difference is that the target keywords are known and predefined. To this end, we propose a new method based on metric learning that maximises the distance between target and non-target keywords, but also learns per-class weights for target keywords à la classification objectives. Experiments on the Google Speech Commands dataset show that our method significantly reduces false alarms to unseen non-target keywords, while maintaining the overall classification accuracy.

Audio and Speech Processing,Sound

An Approach of Keyword Spotting Based on HMM

A Keyword Spotting Method

A New Keyword Spotting Approach for Spontaneous Mandarin Speech

Experimental Investigation into Alignment-based Acoustic Confidence Measures in Keyword Verification for Mandarin Speech

Keyword-specific normalization based keyword spotting for spontaneous speech

A VOCABULARY-INDEPENDENT KEYWORD SPOTTER FOR SPONTANEOUS CHINESE SPEECH

Keyword Spotting Based on Hypothesis Boundary Realignment and State-Level Confidence Weighting

HarkMan—A Vocabulary-Independent Keyword Spotter for Spontaneous Chinese Speech

Keyword Spotting Based on Phoneme Confusion Matrix

A Two-Step Keyword Spotting Method Using Fuzzy Search Algorithm

Keyword Spotting Based on Syllable Confusion Network.

A Multi-oriented Chinese Keyword Spotter Guided by Text Line Detection

An utterance verification algorithm in keyword spotting system

A New Framework for Large Vocabulary Keyword Spotting Using Two-Pass Confidence Measure

Keyword spotting based on mixed grammar model

LEXICAL ACCESS-BASED CONFIDENCE MEASURE FOR A SPANISH KEYWORD SPOTTING SYSTEM

A Two-Step Keyword Spotting Method Based on Context-Dependent a Posteriori Probability

MM-KWS: Multi-modal Prompts for Multilingual User-defined Keyword Spotting

Word Spotting Based on a Posterior Measure of Keyword Confidence

Keyword spotting -- Detecting commands in speech using deep learning

Metric Learning for Keyword Spotting