Abstract:Large Language Models (LLMs) have demonstrated substantial potential for error correction in Automatic Speech Recognition (ASR). However, most research focuses on utterances from short-duration speech recordings, which are the predominant form of speech data for supervised ASR training. This paper investigates the effectiveness of LLMs for error correction in full-text generated by ASR systems from longer speech recordings, such as transcripts from podcasts, news broadcasts, and meetings. First, we develop a Chinese dataset for full-text error correction, named ChFT, utilizing a pipeline that involves text-to-speech synthesis, ASR, and error-correction pair extractor. This dataset enables us to correct errors across contexts, including both full-text and segment, and to address a broader range of error types, such as punctuation restoration and inverse text normalization, thus making the correction process comprehensive. Second, we fine-tune a pre-trained LLM on the constructed dataset using a diverse set of prompts and target formats, and evaluate its performance on full-text error correction. Specifically, we design prompts based on full-text and segment, considering various output formats, such as directly corrected text and JSON-based error-correction pairs. Through various test settings, including homogeneous, up-to-date, and hard test sets, we find that the fine-tuned LLMs perform well in the full-text setting with different prompts, each presenting its own strengths and weaknesses. This establishes a promising baseline for further research. The dataset is available on the website.

Language model adaptation based on correction information for interactive speech transcription

Search and Classification Based Language Model Adaptation

Discriminative Language Model Adaptation for Mandarin Broadcast Speech Transcription and Translation

Correction Focused Language Model Training for Speech Recognition

A Language Model Adaptation Approach Based on Text Classification.

A METHOD TO CONSTRUCT AN ADAPTIVE MONGOLIAN SPEECH ACOUSTIC MODEL

A Light-weight contextual spelling correction model for customizing transducer-based speech recognition systems

A New Topic-Based Language Model Adaptation

Language Model Adaptation Based on the Classification of a Trigram's Language Style Feature

Just-in-time Latent Semantic Adaptation on Language Model for Chinese Speech Recognition Using Web Data

Multi-stage Large Language Model Correction for Speech Recognition

Full-text Error Correction for Chinese Speech Recognition with Large Language Model

Integrated Semantic and Phonetic Post-correction for Chinese Speech Recognition

Effective Text Adaptation for LLM-based ASR through Soft Prompt Fine-Tuning

CTC Regularized Model Adaptation for Improving LSTM RNN Based Multi-Accent Mandarin Speech Recognition

Directed Automatic Speech Transcription Error Correction Using Bidirectional LSTM

Translation Model Adaptation for Statistical Machine Translation with Monolingual Topic Information.

User- Oriented Chinese Language Model and Its Machine Learning

Automatically Correcting Large Language Models: Surveying the Landscape of Diverse Automated Correction Strategies

Improving Handwritten Chinese Text Recognition by Unsupervised Language Model Adaptation

Topic Language Model Adaption for Recognition of Homologous Offline Handwritten Chinese Text Image