Abstract:The automatic identification and analysis of pronunciation errors, known as Mispronunciation Detection and Diagnosis (MDD) plays a crucial role in Computer Aided Pronunciation Learning (CAPL) tools such as Second-Language (L2) learning or speech therapy applications. Existing MDD methods relying on analysing phonemes can only detect categorical errors of phonemes that have an adequate amount of training data to be modelled. With the unpredictable nature of the pronunciation errors of non-native or disordered speakers and the scarcity of training datasets, it is unfeasible to model all types of mispronunciations. Moreover, phoneme-level MDD approaches have a limited ability to provide detailed diagnostic information about the error made. In this paper, we propose a low-level MDD approach based on the detection of speech attribute features. Speech attribute features break down phoneme production into elementary components that are directly related to the articulatory system leading to more formative feedback to the learner. We further propose a multi-label variant of the Connectionist Temporal Classification (CTC) approach to jointly model the non-mutually exclusive speech attributes using a single model. The pre-trained wav2vec2 model was employed as a core model for the speech attribute detector. The proposed method was applied to L2 speech corpora collected from English learners from different native languages. The proposed speech attribute MDD method was further compared to the traditional phoneme-level MDD and achieved a significantly lower False Acceptance Rate (FAR), False Rejection Rate (FRR), and Diagnostic Error Rate (DER) over all speech attributes compared to the phoneme-level equivalent.

Preceding word information for predicting speech errors in English as foreign language speech

Improving Speech Recognition Error Prediction for Modern and Off-the-shelf Speech Recognizers

I know how you'll say it: evidence of speaker-specific speech prediction

End-to-end Mispronunciation Detection with Simulated Error Distance

Preceding vowel duration and syllable-final stop voicing: an examination of l2 english production and perception by chinese learners of english

Improvement in Text-Dependent Mispronunciation Detection for English Learners

Semantic interference affects speech production by increasing disfluencies, not errors

Analysis on Mispronunciations in Capt Based on Computational Speech Perception

Cross-Linguistic Trends in Speech Errors: An Analysis of Sub-Lexical Errors in Cantonese

Convergent neural signatures of speech prediction error are a biological marker for spoken word recognition

Comparison of L2 Korean pronunciation error patterns from five L1 backgrounds by using automatic phonetic transcription

What Comes After /f/? Prediction in Speech Derives From Data-Explanatory Processes

Predictive Speech Recognition and End-of-Utterance Detection Towards Spoken Dialog Systems

Unsupervised Discovery of an Extended Phoneme Set in L2 English Speech for Mispronunciation Detection and Diagnosis.

A Preliminary Study on the Inter-Lexical Pauses in L2 English Read by Chinese Students

Pre-fortis Clipping Patterns: A Cross Linguistic Examination of L2 English Production and Perception by Asian Learners of English

Grading the Severity of Mispronunciations in CAPT Based on Statistical Analysis and Computational Speech Perception

Inappropriate Pause Detection In Dysarthric Speech Using Large-Scale Speech Recognition

A Preliminary Study of Prosodic Disambiguation by Chinese EFL Learners.

Phonological Level wav2vec2-based Mispronunciation Detection and Diagnosis Method

Automatic Disfluency Detection from Untranscribed Speech