Abstract:In automatic speech recognition (ASR) research, discriminative criteria have achieved superior performance in DNN-HMM systems. Given this success, the adoption of discriminative criteria is promising to boost the performance of end-to-end (E2E) ASR systems. With this motivation, previous works have introduced the minimum Bayesian risk (MBR, one of the discriminative criteria) into E2E ASR systems. However, the effectiveness and efficiency of the MBR-based methods are compromised: the MBR criterion is only used in system training, which creates a mismatch between training and decoding; the on-the-fly decoding process in MBR-based methods results in the need for pre-trained models and slow training speeds. To this end, novel algorithms are proposed in this work to integrate another widely used discriminative criterion, lattice-free maximum mutual information (LF-MMI), into E2E ASR systems not only in the training stage but also in the decoding process. The proposed LF-MMI training and decoding methods show their effectiveness on two widely used E2E frameworks: Attention-Based Encoder-Decoders (AEDs) and Neural Transducers (NTs). Compared with MBR-based methods, the proposed LF-MMI method: maintains the consistency between training and decoding; eschews the on-the-fly decoding process; trains from randomly initialized models with superior training efficiency. Experiments suggest that the LF-MMI method outperforms its MBR counterparts and consistently leads to statistically significant performance improvements on various frameworks and datasets from 30 hours to 14.3k hours. The proposed method achieves state-of-the-art (SOTA) results on Aishell-1 (CER 4.10%) and Aishell-2 (CER 5.02%) datasets. Code is released.

CRF-based confidence measures of recognized candidates for lattice-based audio indexing

Lattice-based continuous Chinese speech indexing based on mutual information confidence measure

Experimental Investigation into Alignment-based Acoustic Confidence Measures in Keyword Verification for Mandarin Speech

CRF-based Single-stage Acoustic Modeling with CTC Topology.

Integrating Lattice-Free MMI into End-to-End Speech Recognition

Fusing Multiple Systems into a Compact Lattice Index for Chinese Spoken Term Detection.

CRLB-Based Optimization for Combined FISP and PSIF MR Fingerprinting

A Trellis Based Fast Lattice Generating Algorithm

Lattice-based Indexing for Spontaneous Mandarin Speech

Exploiting Prosodic and Lexical Features for Tone Modeling in A Conditional Random Field Framework

Lattice Rescoring Based on Large Ensemble of Complementary Neural Language Models

A Rescoring Approach for Keyword Search Using Lattice Context Information.

Syllable Based Audio Search Using Confusion Network Arc As Indexing Unit

Sequence-level Confidence Classifier for ASR Utterance Accuracy and Application to Acoustic Models

Improved lattice-based spoken document retrieval by directly learning from the evaluation measures

Upgrading CRFS to JRFS and Its Benefits to Sequence Modeling and Labeling.

Handwritten Chinese/Japanese text recognition using semi-Markov conditional random fields

Letter-to-Sound Pronunciation Prediction Using Conditional Random Fields

A Lattice-Reduction-Aided Sphere Decoder for Underwater Acoustic FBMC/OQAM Communications.

Sparse Higher Order Conditional Random Fields for Improved Sequence Labeling.

Automatic Indexing Model Based on Conditional Random Fields