Abstract:Error correction (EC) models play a crucial role in refining Automatic Speech Recognition (ASR) transcriptions, enhancing the readability and quality of transcriptions. Without requiring access to the underlying code or model weights, EC can improve performance and provide domain adaptation for black-box ASR systems. This work investigates the use of large language models (LLMs) for error correction across diverse scenarios. 1-best ASR hypotheses are commonly used as the input to EC models. We propose building high-performance EC models using ASR N-best lists which should provide more contextual information for the correction process. Additionally, the generation process of a standard EC model is unrestricted in the sense that any output sequence can be generated. For some scenarios, such as unseen domains, this flexibility may impact performance. To address this, we introduce a constrained decoding approach based on the N-best list or an ASR lattice. Finally, most EC models are trained for a specific ASR system requiring retraining whenever the underlying ASR system is changed. This paper explores the ability of EC models to operate on the output of different ASR systems. This concept is further extended to zero-shot error correction using LLMs, such as ChatGPT. Experiments on three standard datasets demonstrate the efficacy of our proposed methods for both Transducer and attention-based encoder-decoder ASR systems. In addition, the proposed method can serve as an effective method for model ensembling.

An Efficient Approach of Language Model Applying in ASR Systems

Using Large Language Model for End-to-End Chinese ASR and NER

Large-scale Language Model Rescoring on Long-form Data

Adapting Large Language Model with Speech for Fully Formatted End-to-End Speech Recognition

Application-Agnostic Language Modeling for On-Device ASR

A Comprehensive Solution to Connect Speech Encoder and Large Language Model for ASR

ASR Error Correction using Large Language Models

A Transcription Prompt-based Efficient Audio Large Language Model for Robust Speech Recognition

Leveraging Large Language Models for Exploiting ASR Uncertainty

Language Model Estimation For Optimizing End-To-End Performance Of A Natural Language Call Routing System

A General Procedure for Improving Language Models in Low-Resource Speech Recognition

Language-agnostic Multilingual Modeling

Recent Advances in Speech Language Models: A Survey

A Comparative Study of LLM-based ASR and Whisper in Low Resource and Code Switching Scenario

Unveiling the Potential of LLM-Based ASR on Chinese Open-Source Datasets

Multilingual and Fully Non-Autoregressive ASR with Large Language Model Fusion: A Comprehensive Study

Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition

An Embarrassingly Simple Approach for LLM with Strong ASR Capacity

Multi-Level Modeling Units for End-to-End Mandarin Speech Recognition

Integration of complex language models in ASR and LU systems

Exploring the Integration of Large Language Models into Automatic Speech Recognition Systems: An Empirical Study