Abstract:<p>Deep learning methods for language recognition have achieved promising performances. However, most of the studies focus on frameworks for single types of acoustic features and single tasks. In this paper, we propose the deep joint learning strategies based on the Multi-Feature (MF) and Multi-Task (MT) models. First, we investigate the efficiency of integrating multiple acoustic features and explore two kinds of training constraints, one is introducing auxiliary classification constraints with adaptive weights for loss functions in feature encoder sub-networks, and the other option is the Canonical Correlation Analysis (CCA) constraint to maximize the correlation of different feature representations. Correlated speech tasks, such as phoneme recognition, are applied as auxiliary tasks in order to learn related information to enhance the performance of language recognition. We analyze phoneme-aware information from different learning strategies, like joint learning on the frame-level, adversarial learning on the segment-level, and the combination mode. In addition, we present the Language-Phoneme embedding extraction structure to learn and extract language and phoneme embedding representations simultaneously. We demonstrate the effectiveness of the proposed approaches with experiments on the Oriental Language Recognition (OLR) data sets. Experimental results indicate that joint learning on the multi-feature and multi-task models extracts instinct feature representations for language identities and improves the performance, especially in complex challenges, such as noisy or cross-channel conditions.</p>

Location-based End-to-end Speech Recognition with Multiple Language Models

Unveiling the Potential of LLM-Based ASR on Chinese Open-Source Datasets

Automatic Call Routing with Multiple Language Models

Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition

Adapting Large Language Model with Speech for Fully Formatted End-to-End Speech Recognition

Acoustic Model Fusion for End-to-end Speech Recognition

Multi-stage Large Language Model Correction for Speech Recognition

Advancing Multi-talker ASR Performance with Large Language Models

Integrating Pre-Trained Speech and Language Models for End-to-End Speech Recognition

Enhancing CTC-based speech recognition with diverse modeling units

Using Large Language Model for End-to-End Chinese ASR and NER

Deep joint learning for language recognition

Multi-task Recurrent Model for True Multilingual Speech Recognition

End-to-end Modeling for Chinese Dialect Speech Recognition

A Transcription Prompt-based Efficient Audio Large Language Model for Robust Speech Recognition

Exploring the Integration of Large Language Models into Automatic Speech Recognition Systems: An Empirical Study

Towards Language-Universal Mandarin-English Speech Recognition

Multi-Level Modeling Units for End-to-End Mandarin Speech Recognition

Improving Mandarin End-to-End Speech Recognition with Word N-gram Language Model

Joint Spoken Language Understanding And Domain Adaptive Language Modeling

ASR Error Correction using Large Language Models