Abstract:In automatic speech recognition (ASR) research, discriminative criteria have achieved superior performance in DNN-HMM systems. Given this success, the adoption of discriminative criteria is promising to boost the performance of end-to-end (E2E) ASR systems. With this motivation, previous works have introduced the minimum Bayesian risk (MBR, one of the discriminative criteria) into E2E ASR systems. However, the effectiveness and efficiency of the MBR-based methods are compromised: the MBR criterion is only used in system training, which creates a mismatch between training and decoding; the on-the-fly decoding process in MBR-based methods results in the need for pre-trained models and slow training speeds. To this end, novel algorithms are proposed in this work to integrate another widely used discriminative criterion, lattice-free maximum mutual information (LF-MMI), into E2E ASR systems not only in the training stage but also in the decoding process. The proposed LF-MMI training and decoding methods show their effectiveness on two widely used E2E frameworks: Attention-Based Encoder-Decoders (AEDs) and Neural Transducers (NTs). Compared with MBR-based methods, the proposed LF-MMI method: maintains the consistency between training and decoding; eschews the on-the-fly decoding process; trains from randomly initialized models with superior training efficiency. Experiments suggest that the LF-MMI method outperforms its MBR counterparts and consistently leads to statistically significant performance improvements on various frameworks and datasets from 30 hours to 14.3k hours. The proposed method achieves state-of-the-art (SOTA) results on Aishell-1 (CER 4.10%) and Aishell-2 (CER 5.02%) datasets. Code is released.

A Comparative Study of RPCL and MCE Based Discriminative Training Methods for LVCSR.

Discriminative training of GMM-HMM acoustic model by RPCL learning

Discriminative training of GMM-HMM acoustic model by RPCL type Bayesian Ying-Yang harmony learning

Improvement Comparison of Different Lattice-based Discriminative Training Methods in Chinese-monolingual and Chinese-English-bilingual Speech Recognition

Discriminative method for recurrent neural network language models

Stream Weight Training Based on MCE for Audio-Visual LVCSR

Integrating Lattice-Free MMI into End-to-End Speech Recognition

Regression class selection and speaker adaptation with MLLR in Mandarin continuous speech recognition

An LVCSR Based Automatic Scoring Method in English Reading Tests

Visual Information Assisted Mandarin Large Vocabulary Continuous Speech Recognition

Deep Audio-visual System for Closed-set Word-level Speech Recognition

Mean and Covariance Adaptation Based on Minimum Classification Error Linear Regression for Continuous Density HMMs

Regularized Minimum Class Variance Extreme Learning Machine for Language Recognition

Minimizing Sequential Confusion Error in Speech Command Recognition

Discriminative Boosting Regression Backend for Phonotactic Language Recognition

Mandarin-English bilingual phone modeling and combining MPE based Discriminative training for cross-language speech recognition

Discriminative Fine-tuning of LVLMs

A Study and Improvement of Minimum Sample Risk Methods for Language Modeling

Multi-View Lstm Language Model With Word-Synchronized Auxiliary Feature For Lvcsr

Language-specific Acoustic Boundary Learning for Mandarin-English Code-switching Speech Recognition

Contrastive Learning With Audio Discrimination For Customizable Keyword Spotting In Continuous Speech