Abstract:Spoken Language Understanding (SLU) aims to interpret the meanings of human speeches in order to support various human-machine interaction systems. A key technique for SLU is Automatic Speech Recognition (ASR), which transcribes speech signals into text contents. As the output texts of modern ASR systems unavoidably contain errors, mainstream SLU models either trained or tested on texts transcribed by ASR systems would not be sufficiently error robust. We present ARoBERT, an ASR Robust BERT model, which can be fine-tuned to solve a variety of SLU tasks with noisy inputs. To guarantee the robustness of ARoBERT, during pretraining, we decrease the fluctuations of language representations when some parts of the input texts are replaced by homophones or synophones. Specifically, we propose two novel self-supervised pre-training tasks for ARoBERT, namely Phonetically-aware Masked Language Modeling (PMLM) and ASR Model-adaptive Masked Language Modeling (AMMLM). The PMLM task explicitly fuses the knowledge of word phonetic similarities into the pre-training process, which forces homophones and synophones to share similar representations. In AMMLM, a data-driven algorithm is further introduced to mine typical ASR errors such that ARoBERT can tolerate ASR model errors. In the experiments, we evaluate ARoBERT over multiple datasets. The results show the superiority of ARoBERT, which consistently outperforms strong baselines. We have also shown that ARoBERT outperforms state-of-the-arts on a public benchmark. Currently, ARoBERT has been deployed in an online production system with significant improvements.

Robust Spoken Language Understanding with Acoustic and Domain Knowledge

Robust Spoken Language Understanding With Unsupervised Asr-Error Adaptation

Robust Spoken Language Understanding with RL-based Value Error Recovery

CATSLU: the 1st Chinese Audio-Textual Spoken Language Understanding Challenge

A Joint and Domain-Adaptive Approach to Spoken Language Understanding

Multi-Domain Spoken Language Understanding Using Domain- and Task-Aware Parameterization

Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networks

ML-LMCL: Mutual Learning and Large-Margin Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding

Multimodal Audio-textual Architecture for Robust Spoken Language Understanding

Joint Spoken Language Understanding And Domain Adaptive Language Modeling

Transfer Learning Methods for Spoken Language Understanding

ARoBERT: An ASR Robust Pre-Trained Language Model for Spoken Language Understanding

Modality Confidence Aware Training for Robust End-to-End Spoken Language Understanding

End-to-End Cross-Lingual Spoken Language Understanding Model with Multilingual Pretraining.

How ChatGPT is Robust for Spoken Language Understanding?

A Survey on Spoken Language Understanding: Recent Advances and New Frontiers.

A Survey on Speech Large Language Models

Multi-Classification Model for Spoken Language Understanding

DiscreteSLU: A Large Language Model with Self-Supervised Discrete Speech Units for Spoken Language Understanding

Improving Spoken Language Understanding By Exploiting ASR N-best Hypotheses

A Weakly Supervised Learning Approach for Spoken Language Understanding.