ASR-based Mandarin Pronunciation Teaching System

Shenghui Zhao

2005-01-01

Abstract:The paper reviews ASR-based pronunciation teaching system and describes the basis idea and the recent research progress of it,some key technologies are discussed in detail,and some thoughts on ASR-based Mandarin pronunciation tea￣ching system are also given.

What problem does this paper attempt to address?

A Mobile Learning System For Learning Mandarin Pronunciation

Long Zhang,Jianhua Wang,Haifeng Li

DOI: https://doi.org/10.1109/ICCSNT.2011.6182034

2012-01-01

Abstract:Based on speech recognition technology and wireless communication technology, a novel mobile learning system for learning Mandarin pronunciation is built. It likes an excellent human teacher to help learners improve their pronunciation quality and has the following good features: intelligence, interaction, and convenience. After training the standard acoustic model, the system can recognize the learner's speech and then calculate the accuracy rate of the pronunciation. Compared the practical pronunciation with the standard pronunciation, the system can accurately detect the defects and errors of the learner's pronunciation, and locate its position and type. Combined the phonetics knowledge and expert knowledge, the system also can give a concrete pronunciation diagnosis, and make reasonable suggestions and correction methods. Finally, Using intelligent search techniques, adaptive training materials can be automatically generated and reinforcement learning will be performed.
Automatic Mandarin Pronunciation Scoring for Native Learners with Dialect Accent.

Si Wei,Qing-Sheng Liu,Yu Hu,Ren-Hua Wang

DOI: https://doi.org/10.21437/interspeech.2006-416

2006-01-01

Abstract:This paper studies pronunciation Scoring algorithm in CALL system aiming at teaching native Chinese learn standard Mandarin. Most of the pronunciation scoring algorithms focus on non-native environment, which may not be suitable for native speakers. We bring up a new algorithm based on traditional posterior log-likelihood algorithm by weighting the initial part of Mandarin syllables, where final-initial's duration ratio is introduced to control the weight. Experiments show that the proposed algorithm is much more effective than traditional posterior log-likelihood algorithm in the Mandarin learning system. The correlation with human score achieves an increase of 11%.
Pronunciation Evaluation Technology Based on Computer Aided Chinese Learning System

Huangxian Ju

2014-01-01

Computer Engineering and Applications Journal

Abstract:A method for automatic pronunciation evaluation is proposed, applying to the learners with Mandarin as a second language learning Chinese. Bilingual language speech database is introduced and the evaluation is divided into two parts, which are the manner of articulation and pronunciation. This paper extracts the parameters of each part and makes use of the improved method based on comparison of speech similarity, an algorithm named Bilingual Dynamic Time Warping. It gets the final result by means of score mechanism. Experiments verificate the validity of the system.
An Unified and Automatic Approach of Mandarin HTS System.

Yong Guan,Jilei Tian,Yi-Jian Wu,Junichi Yamagishi,Jani Nurminen

2010-01-01

Abstract:Most studies on Mandarin HTS (HMM-based text-to-speech system) have taken the initial/final as the basic acoustic units. It is, however, challenging to develop a multilingual HTS in a uniformed and consistent way since most of other languages use the phoneme as the basic phonetic unit. It becomes hard to apply cross-lingual adaptation which need map phonemes from each other, particularly in the case of unified ASR and HTS system due to the phoneme nature of most of the ASR systems. In this paper, we propose a phoneme based Mandarin HTS system, which has been systematically evaluated by comparing it with the initial/final system. The experimental results show that the use of phoneme as the acoustic unit for Mandarin HTS is a promising unified approach, thus enabling better and more uniform development with other languages while significantly reducing the number of acoustic units. The flat-start training scheme is also evaluated to show that the phoneme segmentation problem is solved without any performance degradation for phoneme based Mandarin HTS system. This performs an automatic approach without dependency with particular ASR system.
A Spoken English Teaching System Based on Speech Recognition and Machine Learning

Fengming Jiao,Jiao Song,Xin Zhao,Ping Zhao,Ru Wang

DOI: https://doi.org/10.3991/ijet.v16i14.24049

2021-07-28

International Journal of Emerging Technologies in Learning (iJET)

Abstract:The learning model and environment are two major constraints on spoken English learning by Chinese learners. The maturity of computer-aided language learning brings a new opportunity to spoken English learners. Based on speech recognition and machine learning, this paper designs a spoken English teaching system, and determines the overall architecture and functional modules of the system according to the system’s functional demand. Specifically, MATLAB was adopted to realize speech recognition, and generate a speech recognition module. Combined with machine learning algorithm, a deep belief network (DBN)-support vector machine (SVM) model was proposed to classify and detect the errors in pronunciation; the module also scores the quality and corrects the errors in pronunciation. This model was extended to a speech evaluation module was created. Next, several experiments were carried out to test multiple attributes of the system, including the accuracy of pronunciation classification and error detection, recognition rates of different environments and vocabularies, and the real-timeliness of recognition. The results show that our system achieved good performance, realized the preset design goals, and satisfied the user demand. This research provides an important theoretical and practical reference to transforming English teaching method, and improving the spoken English of learners.
Main vowel domain tone modeling with lexical and prosodic analysis for Mandarin ASR

Shilei Zhang,Shi, Qin,Chu, S.M.,Yong Qin

DOI: https://doi.org/10.1109/ICASSP.2009.4960645

2009-01-01

Abstract:The tone is a distinctive discriminative feature in Mandarin Chinese. Often functional, yet seldom thorough are most large-scale Mandarin speech recognition systems in treating tone modeling. In particular, many lack the necessary sophistication to deal with the myriad variations arising from the combination of acoustic and lexical contexts. This paper reports an attempt to account for these variabilities and to bring richer tone modeling into the IBM Mandarin broadcast transcription system. In particular, we describe a system that combines the embedded approach and a novel explicit tone modeling technique characterized by a. robust tone tracking in the main-vowel domain, and b. context-dependent models with lexical and prosodic contexts. The proposed method is validated on a connected-digits set and subsequently evaluated on a large-vocabulary broadcast transcription task. It is shown that 14.8% and 5.4% relative reductions in character error rate are achieved respectively.
Research on pronunciation scoring technology in language training system

Zhiyuan Liu,Fuan Wen,Zhihong Lu,Meisong Chen

DOI: https://doi.org/10.1109/ICeEEM.2011.6137788

2011-01-01

Abstract:Due to many foreign language learners do not have speaking enough practice opportunities, we have designed spoken pronunciation training software based on speech recognition technology. This voice recognition software module can score the spoken words based on the pronunciation of each user. It solve the difficult problem of spoken language training. In this paper, we will detail the specific design of this oral practice, key technologies and implementation methods. © 2011 IEEE.
Automatic Mispronunciation Detection for Mandarin Chinese

ZHANG Feng,HUANG Chao,DAI Lirong

DOI: https://doi.org/10.3969/j.issn.1003-0077.2010.02.015

2010-01-01

Abstract:The current automatic mispronunciation detection systems are mostly based on automatic speech recognition(ASR) framework with statistical model.This paper presents the methods to improve the performance of mispronunciation detection at syllable level for Mandarin Chinese from two aspects: introducing the speaker adaptive training(SAT) and the selective maximum likelihood linear regression(SMLLR) to get a better acoustic statistical model,and proposing speaker normalization backend because of the limited information and the different rating level for the different pronunciation level.Experiments on a database of 8000 syllables pronounced by 40 speakers with varied pronunciation proficiency indicate the promising effects of these strategies by improving the precision from 45.8% to 53.6% at 30% recall,and 64.6% to 79.9% at 10% recall.
STUDY ON FRAMEWORK FOR CHINESE PRONUNCIATION VARIATION MODELING

Jing Li,Mingxing Xu,Wenhu Wu

2002-01-01

Abstract:The pronunciation variations, which badly influenced the performance of ASR system, are serious in continuous speech, especially in spontaneous speech. Many research works are focused on pronunciation variation modeling in recent years. A framework for Chinese pronunciation variation modeling is described in this paper. The main idea is that the pronunciation variations are hidden in the recognition errors and it could be found out and modeled. The linguistic knowledge can be used for supervising the finding variations, and it can be used to model the variations directly. Under this framework, the pronunciation variation modeling can be made more reliably and more expediently than conventional methods. Our primary experimental results show it is effective for Chinese continuous speech recognition.
An Mandarin Pronunciation Quality Assessment System Using Two Kinds of Acoustic Models

Fengpei Ge,Li Lu,Changliang Liu,Fuping Pan,Bin Dong,Yonghong Yan

DOI: https://doi.org/10.1109/icrccs.2009.25

2009-01-01

Abstract:This paper presents our Mandarin pronunciation quality assessment system for the examination of Putonghua Shuiping Kaoshi (PSK) and investigates some measures to improve the assessment accuracy. In this paper, a selective speaker adaptation method is studied. In the adaptation module, we select well pronounced speech as the adaptation data, and adopt Maximum Likelihood Linear Regression (MLLR) to update the speaker-independent (SI) acoustic model. Besides the triphone based acoustic model, the monophone based acoustic model is also applied to our system. Further improvements are obtained by combining posterior probabilities computed with triphone and monophone based acoustic models using Support Vector Machine (SVM) to assess the goodness of pronunciations. The experiment results show that the average correlation coefficient (ACC) between machine and the human scores achieves 0.8549, almost equivalent to ACC between different experts. The improved system achieves usable performance in actual applications.
ASR-Based Input Method for Postal Address Recognition in Chinese Mandarin

Ling Feng Wei,Sun Maosong

DOI: https://doi.org/10.1007/978-3-319-12277-9_27

2014-01-01

Abstract:As the automatic speech recognition technology (ASR) has becoming more and more mature, especially with statistical language modeling built with web scale data, and with the utilization of Hidden Markov Model probabilistic framework, speech recognition has become applicable to many domains and usage scenarios. In particular, speech recognition can be applied to task such as Chinese postal address recognition. This paper presents the first attempt ever, in both academic and commercial settings, to create an ASR-based input method for postal address recognition in Chinese Mandarin. By customizing the statistical language model to such domain, and incorporating the knowledge from the structural information provided by geo-topology, our language model successfully captures the signals from geographical contextual information and self-correct possible mis-recognitions. Experiment results provide evident that our approach based on speech recognition achieves a faster and a more accuracy input method compare to traditional keyboard-based input.
Standardized Evaluation Method of Pronunciation Teaching Based on Deep Learning

Xiaoda Zhao,Xiaoyan Jin

DOI: https://doi.org/10.1155/2022/8961836

IF: 1.968

2022-03-07

Security and Communication Networks

Abstract:With the advancement of globalization, an increasing number of people are learning and using a common language as a tool for international communication. However, there are clear distinctions between the native language and target language, especially in pronunciation, and the domestic target language, the learning environment is far from ideal, with few competent teachers. In addition, such learning cannot achieve computer-assisted language learning (CALL) technology. The efficient combination of computer technology and language teaching and learning methods provides a new solution to this problem. The core of CALL is speech recognition (SR) technology and speech evaluation technology. The development of deep learning (DL) has greatly promoted the development of speech recognition. The pronunciation resource collected from the Chinese college students, whose majors are language education or who are planning to obtain better pronunciation, shall be the research object of this paper. The study applies deep learning to the standard but of target language pronunciation and builds a standard evaluation model of pronunciation teaching based on the deep belief network (DBN). On this basis, this work improves the traditional pronunciation quality evaluation method, comprehensively considers intonation, speaking speed, rhythm, intonation, and other multi-parameter indicators and their weights, and establishes a reasonable and efficient pronunciation model. The systematic research results show that this article has theoretical and practical value in the field of phonetics education.

computer science, information systems,telecommunications
An English Pronunciation Training System Based on Speech Comparison Technology

Zhuo-yan JIA,Rui-min SHEN

DOI: https://doi.org/10.3969/j.issn.1006-9348.2005.04.079

2005-01-01

Abstract:Foreign language learning is a difficult and time-consuming task. It i s not affordable for most students to be in one-to-one interactions with a teach er who is a native speaker. They can mimic native speakers' utterance from tapes and discs, but they have to check the imitation result by themselves. In most c ases, they could not recognize their mistakes or weakness in their pronunciation . An English pronunciation training system is introduced in this paper. This sys tem can help users to find their problems in pronunciation and even locate the w orst pronounced syllable, and provide the corresponding guide.
ASR-based system for promoting pronunciation

Sariani Sariani,Mutia El Khairat,Welsi Haslina,Baety Baetty

DOI: https://doi.org/10.1075/aila.23021.sar

2024-06-13

AILA Review

Abstract:Abstract In developing English as a Lingua Franca, educators and researchers must employ new methods in language acquisition to make the learners internationally intelligible and comprehensible. This study aimed to determine the implication of infinite access to Automatic Speech Recognition (ASR)-based language-learning program promoting pronunciation skills acquisition of vocational higher education students. In this study, the students learned English as Lingua Franca (ELF) through a collaborative approach. It applied a qualitative approach with 67 first-year university students from three intact classes as participants. One class of 24 students was assigned as the first group utilising the ASR BoldVoice as the additional learning materials in their speaking class. In contrast, the other two classes of 43 students were set as the second and third groups carrying out conventional learning procedures. The development of participants’ utterances was analysed in terms of fluency, completeness, and accuracy at pre- and post-test. In order to support the obtained data, a semi-structured interview was performed right after the post-test. The result showed that the pronunciation skill of the experimental group was significantly improved in particular phones of pronunciation: /ch/, /th/, /sh/, and /j/. The analysis of the interview data confirmed that the students gained substantial improvement with regard to their pronunciation and communicative competence compared to the conventional learning process.
Research On the Chinese initial pronunciation computer-aided language learning system

Gang LIU,Datian YE

DOI: https://doi.org/10.3969/j.issn.1002-3208.2008.02.014

2008-01-01

Abstract:In order to facilitate the language rehabilitation of the hearing impaired person,an automatic speech recognition-based computer-aided language learning(CALL)system is designed.Firstly,an utterance database produced by a standard spoken teacher is established and the key pronunciation mouth shape frames are recorded.Secondly,the static and dynamic features are extracted from the utterance database and we apply the continuous density hidden markov model(CDHMM) to imitate Chinese initial,which is trained with embedded training algorithm and performs recognition task with token passing algorithm and recognition network.The outside test recognition rate of the teacher pronunciation is 96.65%,which means that the system can reasonably work as the benchmark system of the Chinese initial pronunciation learning.Finally,the performance comparison experiment based on visual feedback proves that the system can aid the deaf-dumb to recover language ability effectively.
M2ASR: Ambitions and First Year Progress

Dong Wang,Thomas Fang Zheng,Zhiyuan Tang,Ying Shi,Lantian Li,Shiyue Zhang,Hongzhi Yu,Guanyu Li,Shipeng Xu,Askar Hamdulla,Mijit Ablimit,Gulnigar Mahmut

DOI: https://doi.org/10.1109/icsda.2017.8384469

2017-01-01

Abstract:In spite of the rapid development of speech techniques, most of the present achievements are for a few major languages, e.g., English and Chinese. Unfortunately, most of the languages in the world are 'minority languages', in the sense that they are spoken by a small population and with limited resource accumulation. Since the present speech technologies are mostly based on big data, partly due to the profound impact of deep learning, they are not directly applicable to minority languages. However, minority languages are so numerous and important that if we want to break the language barrier, they must be seriously taken into account. Recently, the Chinese government approved a fundamental research for minority languages in China: Multilingual Minorlingual Automatic Speech Recognition (M2ASR). Although the initial goal was speech recognition, the ambition of this project is more than that: it intends to publish all the achievements and make them free for the research community, including speech and text corpora, phone sets, lexicons, tools, recipes and prototype systems. In this paper, we will describe this project, report the first-year progress, and present the future plan.
Modeling Syllable-Based Pronunciation Variation for Accented Mandarin Speech Recognition

Shilei Zhang,Qin Shi,Yong Qin

DOI: https://doi.org/10.1109/icpr.2010.397

IF: 8

2010-01-01

Pattern Recognition

Abstract:Pronunciation variation is a natural and inevitable phenomenon in an accented Mandarin speech recognition application. In this paper, we integrate knowledge-based and data-driven approaches together for syllable-based pronunciation variation modeling to improve the performance of Mandarin speech recognition system for speakers with Southern accent. First, we generate the syllable-based pronunciation variation rules of Southern accent observed from the training corpus by Chinese linguistic expert. Second, dictionary augmentation with multiple pronunciation variants and pronunciation probability derived from forced alignment statistics of training data. The acoustic models will be retrained based on the new expansion dictionary. Finally, pronunciation variation adaptation will be performed to further fit the data on the decoding stage by taking distribution of variation rules clusters of testing set into account. The experimental results show that the proposed method provides a flexible framework to improve the recognition performance for accented speech effectively.
The impact of automatic speech recognition technology on second language pronunciation and speaking skills of EFL learners: a mixed methods investigation

Weina Sun

DOI: https://doi.org/10.3389/fpsyg.2023.1210187

IF: 3.8

2023-08-17

Frontiers in Psychology

Abstract:Introduction: This study employed an explanatory sequential design to examine the impact of utilizing automatic speech recognition technology (ASR) with peer correction on the improvement of second language (L2) pronunciation and speaking skills among English as a Foreign Language (EFL) learners. The aim was to assess whether this approach could be an effective tool for enhancing L2 pronunciation and speaking abilities in comparison to traditional teacher-led feedback and instruction. Methods: A total of 61 intermediate-level Chinese EFL learners were randomly assigned to either a control group (CG) or an experimental group (EG). The CG received conventional teacher-led feedback and instruction, while the EG used ASR technology with peer correction. Data collection involved read-aloud tasks, spontaneous conversations, and IELTS speaking tests to evaluate L2 pronunciation and speaking skills. Additionally, semi-structured interviews were conducted with a subset of the participants to explore their perceptions of the ASR technology and its impact on their language learning experience. Results: The quantitative analysis of the collected data demonstrated that the EG outperformed the CG in all measures of L2 pronunciation, including accentedness and comprehensibility. Furthermore, the EG exhibited significant improvements in global speaking skill compared to the CG. The qualitative analysis of the interviews revealed that the majority of the participants in the EG found the ASR technology to be beneficial in enhancing their L2 pronunciation and speaking abilities. Discussion: The results of this study suggest that the utilization of ASR technology with peer correction can be a potent approach in enhancing L2 pronunciation and speaking skills among EFL learners. The improved performance of the EG compared to the CG in pronunciation and speaking tasks demonstrates the potential of incorporating ASR technology into language learning environments. Additionally, the positive feedback from the participants in the EG underscores the value of using ASR technology as a supportive tool in language learning classrooms.

psychology, multidisciplinary
Pronunciation Variation Modeling For Mandarin With Accent

Zhang Chi,Wu Ji,Xiao Xi,Wang Zuoying

DOI: https://doi.org/10.21437/interspeech.2006-246

2006-01-01

Abstract:In order to solve the problem of the performance decrease when state-of-art automatic speech recognition (ASR) system facing accent speech, we propose the Pronunciation Variation Model (PVM). Two approaches are proposed to construct the PVM in this paper. 6.38% and 7.78% relative error rate reduction is achieved for Shanghai and Wuhan accent mandarin, respectively. The experiment on these two typical accent mandarin shows it is a possible way to deal with accent speech.
Pronunciation Modeling of Foreign Words for Mandarin ASR by Considering the Effect of Language Transfer

Lei Wang,Rong Tong

DOI: https://doi.org/10.48550/arXiv.2210.03603

2022-10-07

Computation and Language

Abstract:One of the challenges in automatic speech recognition is foreign words recognition. It is observed that a speaker's pronunciation of a foreign word is influenced by his native language knowledge, and such phenomenon is known as the effect of language transfer. This paper focuses on examining the phonetic effect of language transfer in automatic speech recognition. A set of lexical rules is proposed to convert an English word into Mandarin phonetic representation. In this way, a Mandarin lexicon can be augmented by including English words. Hence, the Mandarin ASR system becomes capable to recognize English words without retraining or re-estimation of the acoustic model parameters. Using the lexicon that derived from the proposed rules, the ASR performance of Mandarin English mixed speech is improved without harming the accuracy of Mandarin only speech. The proposed lexical rules are generalized and they can be directly applied to unseen English words.

ASR-based Mandarin Pronunciation Teaching System

A Mobile Learning System For Learning Mandarin Pronunciation

Automatic Mandarin Pronunciation Scoring for Native Learners with Dialect Accent.

Pronunciation Evaluation Technology Based on Computer Aided Chinese Learning System

An Unified and Automatic Approach of Mandarin HTS System.

A Spoken English Teaching System Based on Speech Recognition and Machine Learning

Main vowel domain tone modeling with lexical and prosodic analysis for Mandarin ASR

Research on pronunciation scoring technology in language training system

Automatic Mispronunciation Detection for Mandarin Chinese

STUDY ON FRAMEWORK FOR CHINESE PRONUNCIATION VARIATION MODELING

An Mandarin Pronunciation Quality Assessment System Using Two Kinds of Acoustic Models

ASR-Based Input Method for Postal Address Recognition in Chinese Mandarin

Standardized Evaluation Method of Pronunciation Teaching Based on Deep Learning

An English Pronunciation Training System Based on Speech Comparison Technology

ASR-based system for promoting pronunciation

Research On the Chinese initial pronunciation computer-aided language learning system

M2ASR: Ambitions and First Year Progress

Modeling Syllable-Based Pronunciation Variation for Accented Mandarin Speech Recognition

The impact of automatic speech recognition technology on second language pronunciation and speaking skills of EFL learners: a mixed methods investigation

Pronunciation Variation Modeling For Mandarin With Accent

Pronunciation Modeling of Foreign Words for Mandarin ASR by Considering the Effect of Language Transfer