Abstract:In recent decades, there has been a great deal of research into the problem of bilingual speech recognition - to develop a recognizer that can handle inter- and intra-sentential language switching between two languages. This paper presents our recent work on the development of a grammar-constrained, Mandarin-English bilingual Speech Recognition System (MESRS) for real world music retrieval. Two of the main difficult issues in handling the bilingual speech recognition systems for real world applications are tackled in this paper. One is to balance the performance and the complexity of the bilingual speech recognition system; the other is to effectively deal with the matrix language accents in embedded language**. In order to process the intra-sentential language switching and reduce the amount of data required to robustly estimate statistical models, a compact single set of bilingual acoustic models derived by phone set merging and clustering is developed instead of using two separate monolingual models for each language. In our study, a novel Two-pass phone clustering method based on Confusion Matrix (TCM) is presented and compared with the log-likelihood measure method. Experiments testify that TCM can achieve better performance. Since potential system users' native language is Mandarin which is regarded as a matrix language in our application, their pronunciations of English as the embedded language usually contain Mandarin accents. In order to deal with the matrix language accents in embedded language, different non-native adaptation approaches are investigated. Experiments show that model retraining method outperforms the other common adaptation methods such as Maximum A Posteriori (MAP). With the effective incorporation of approaches on phone clustering and non-native adaptation, the Phrase Error Rate (PER) of MESRS for English utterances was reduced by 24.47% relatively compared to the baseline monolingual English system while the PER on Mandarin utterances was comparable to that of the baseline monolingual Mandarin system. The performance for bilingual utterances achieved 22.37% relative PER reduction.

Study on Performance Optimization for Chinese Speech Retrieval

Query Expansion Based High Performance Chinese Voice Retrieval

Research on Inter-Syllable Context-Dependent Acoustic Unit for Mandarin Continuous Speech Recognition.

Acoustic Modeling Research of Chinese-English Bilingual Isolated Word Recognition System

Research on Modeling Units of Transformer Transducer for Mandarin Speech Recognition

Improved Semantic Retrieval of Spoken Content by Language Models Enhanced with Acoustic Similarity Graph

A Mandarin Speech Recognition Result Evaluation Algorithm on Mixed Words

A comparable study of modeling units for end-to-end Mandarin speech recognition

A Novel Hybrid Mandarin Speech Synthesis System Using Different Base Units for Model Training and Concatenation

Investigation of Modeling Units for Mandarin Speech Recognition Using Dfsmn-ctc-smbr

Automatic segmentation of Chinese Mandarin speech into syllable-like

A Method for Syllable Segmentation in Mandarin Speech Recognition

Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Study

Development of A Mandarin-English Bilingual Speech Recognition System for Real World Music Retrieval

Research on speech recognition models in the Chinese dictation machine

High Performance Digit Mandarin Speech Recognition

Improving the Syllable-Synchronous Network Search Algorithm for Word Decoding in Continuous Chinese Speech Recognition

Algorithm for Mandarin Continuous Speech Recognition Based on Context-Dependent Unit Between Syllables

Research on continuous Chinese speech recognition system and knowledge based search strategies

Multi-Level Modeling Units for End-to-End Mandarin Speech Recognition

Improved Semantic Retrieval of Spoken Content by Document/Query Expansion with Random Walk Over Acoustic Similarity Graphs