Abstract:Automatic speech Recognition (ASR) is a fundamental and important task in the field of speech and natural language processing. It is an inherent building block in many applications such as voice assistant, speech translation, etc. Despite the advancement of ASR technologies in recent years, it is still inevitable for modern ASR systems to have a substantial number of erroneous recognition due to environmental noise, ambiguity, etc. Therefore, the error correction in ASR is crucial. Motivated by this, this paper studies ASR error correction in the Chinese language, which is one of the most popular languages and enjoys a large number of users in the world. We first create a benchmark dataset named \emph{ASR-EC} that contains a wide spectrum of ASR errors generated by industry-grade ASR systems. To the best of our knowledge, it is the first Chinese ASR error correction benchmark. Then, inspired by the recent advances in \emph{large language models (LLMs)}, we investigate how to harness the power of LLMs to correct ASR errors. We apply LLMs to ASR error correction in three paradigms. The first paradigm is prompting, which is further categorized as zero-shot, few-shot, and multi-step. The second paradigm is finetuning, which finetunes LLMs with ASR error correction data. The third paradigm is multi-modal augmentation, which collectively utilizes the audio and ASR transcripts for error correction. Extensive experiments reveal that prompting is not effective for ASR error correction. Finetuning is effective only for a portion of LLMs. Multi-modal augmentation is the most effective method for error correction and achieves state-of-the-art performance.

ACOUSTIC LEVEL ERROR ANALYSIS IN CONTINUOUS SPEECH RECOGNITION

How Noise and Language Proficiency Influence Speech Recognition by Individual Non-Native Listeners.

Experimental Investigation into Alignment-based Acoustic Confidence Measures in Keyword Verification for Mandarin Speech

Experiments and Analysis for Speaker Dependent Mandarin Syllable Recognition.

Lightly supervised acoustic model training for mandarin continuous speech recognition

Effective Acoustic Modeling for Pronunciation Quality Scoring of Strongly Accented Mandarin Speech

Research on Inter-Syllable Context-Dependent Acoustic Unit for Mandarin Continuous Speech Recognition.

Error Correction by Paying Attention to Both Acoustic and Confidence References for Automatic Speech Recognition

Research on Context-Dependent Acoustical Unit (Triphone) for Mandarin Continuous Speech Recognition

Noisy speech recognition performance of discriminative HMMs

Two-level Search Strategy Based on Reverse HMM and Its Applications

A Fast Error-tolerant Algorithm in Decoding Module of Speech Recognition

STUDY ON FRAMEWORK FOR CHINESE PRONUNCIATION VARIATION MODELING

Estimating and detecting method and system for telephone continuous speech recognition system performance

ASR-EC Benchmark: Evaluating Large Language Models on Chinese ASR Error Correction

Research on continuous Chinese speech recognition system and knowledge based search strategies

Research on Optimal Algorithm of Likelihood Computation in Continuous Speech Recognition

Automatic Error Detection For Unit Selection Speech Synthesis Using Log Likelihood Ratio Based Svm Classifier

TONE RECOGNITION OF CHINESE CONTINUOUS SPEECH

RESEARCH AND REALISATION OF TANDEM IN MANDARIN SPEECH ERROR DETECTION SYSTEM

ON CHINESE CONTINUOUS SPEECH RECOGNITION SYSTEM AND KNOWLEDGE BASED SEARCH STRATEGIES SONG Zhanjiang