Abstract:In recent decades, there has been a great deal of research into the problem of bilingual speech recognition - to develop a recognizer that can handle inter- and intra-sentential language switching between two languages. This paper presents our recent work on the development of a grammar-constrained, Mandarin-English bilingual Speech Recognition System (MESRS) for real world music retrieval. Two of the main difficult issues in handling the bilingual speech recognition systems for real world applications are tackled in this paper. One is to balance the performance and the complexity of the bilingual speech recognition system; the other is to effectively deal with the matrix language accents in embedded language**. In order to process the intra-sentential language switching and reduce the amount of data required to robustly estimate statistical models, a compact single set of bilingual acoustic models derived by phone set merging and clustering is developed instead of using two separate monolingual models for each language. In our study, a novel Two-pass phone clustering method based on Confusion Matrix (TCM) is presented and compared with the log-likelihood measure method. Experiments testify that TCM can achieve better performance. Since potential system users' native language is Mandarin which is regarded as a matrix language in our application, their pronunciations of English as the embedded language usually contain Mandarin accents. In order to deal with the matrix language accents in embedded language, different non-native adaptation approaches are investigated. Experiments show that model retraining method outperforms the other common adaptation methods such as Maximum A Posteriori (MAP). With the effective incorporation of approaches on phone clustering and non-native adaptation, the Phrase Error Rate (PER) of MESRS for English utterances was reduced by 24.47% relatively compared to the baseline monolingual English system while the PER on Mandarin utterances was comparable to that of the baseline monolingual Mandarin system. The performance for bilingual utterances achieved 22.37% relative PER reduction.

Rejection based on a posteriori probability estimated by MLP with application for Mandarin voice dialer on ASIC

A Rejection Model Based on Multi-Layer Perceptrons for Mandarin Digit Recognition

MicPro: Microphone-based Voice Privacy Protection

Real-time Speaker Recognition System for PDA

Integrating Lattice-Free MMI into End-to-End Speech Recognition

Mandarin-English bilingual phone modeling and combining MPE based Discriminative training for cross-language speech recognition

Multi-layer Perceptron for Isolated Mandarin Vocal Number Recognition

Adversarial Post-Processing of Voice Conversion Against Spoofing Detection

The dynamically-adjustable histogram pruning method for embedded voice dialing

A k-climax neighbors policy based viterbi decoding for large vocabulary mandarin speech recognition

High Performance Digit Mandarin Speech Recognition

Improved Posterior Probability Estimation Methods for the Freely-Spoken Speech Evaluation

A Hierarchical Viterbi Algorithm For Mandarin Hybrid Speech Synthesis System

EffectiveASR: A Single-Step Non-Autoregressive Mandarin Speech Recognition Architecture with High Accuracy and Inference Speed

Duration Model for post-processing in a Mandarin speech recognition system

Development of A Mandarin-English Bilingual Speech Recognition System for Real World Music Retrieval

Phone modeling and combining discriminative training for Mandarin-English bilingual speech recognition

Application of the Artificial Intelligence Algorithm in the Automatic Segmentation of Mandarin Dialect Accent

Improving the Decoding Efficiency of Deep Neural Network Acoustic Models by Cluster-Based Senone Selection.

Improvement Comparison of Different Lattice-based Discriminative Training Methods in Chinese-monolingual and Chinese-English-bilingual Speech Recognition

A Novel Efficient Decoding Algorithm for CDHMM-based Speech Recognizer on Chip.