Automatic lexical stress detection using acoustic features for computer-assisted language learning

Junhong Zhao,Hua Yuan,Jia Liu,Shanhong Xia

DOI: https://doi.org/10.26686/wgtn.14524884.v1

2021-01-01

Abstract:No description supplied

What problem does this paper attempt to address?

Automatic Lexical Stress Detection for English Learning

Y Zhu,J Liu,RS Liu

DOI: https://doi.org/10.1109/nlpke.2003.1276001

2003-01-01

Abstract:We examine a method to automatically detect lexical stress of English, which is a part of a computer-assisted language learning (CALL) system for Chinese to learn English pronunciation. Acoustic differences between vowels in syllables that do or do not carry lexical stress are investigated. We use coefficients based on fundamental frequency, energy and duration to differentiate stressed vowels from unstressed ones by linear classification. Although the distributions of the coefficients overlap to some extent, the new normalized pitch-based coefficient obtains correct classification results of about 84%. Compared with previous detection methods, the results suggest that proper normalization of acoustic feature is much more valuable than the combination of different features.
AUTOMATIC STRESS PREDICTION OF CHINESE SPEECH SYNTHESIS

Jianhua Tao,Sheng Zhao,Lian-Hong Cai

2002-01-01

Abstract:The stress was proved to be the essential links between linguistics and acoustics, and behaves as an important parameter for prosody processing and unit selection in speech synthesis system. In the paper, some acoustical measurements are carried out on F0, duration, silence in order to disclose the relationship between stress and acoustical parameters. The normalized compared acoustic parameters are induced to facilitate the stress detecting from the speech. Furthermore, a rule-learning approach is proposed to predict stress in unrestricted Chinese text. In order to improve the accuracy rate of prediction rules, the most effective linguistic features related to stress are selected according to several experiments. The method is proved to be very successful and has been integrated into our speech synthesis system. We get 86% accurate rate of stress prediction. Further listening tests also show that the expressive force of synthesized speech is improved a lot compared to the systems based on traditional method.
English Sentence Stress Detection System Based on HMM Framework

Li Chaolei,Liu Jia,Xia Shanhong

DOI: https://doi.org/10.1016/j.amc.2006.06.081

IF: 4.397

2007-01-01

Applied Mathematics and Computation

Abstract:Computer Assisted Language Learning based on speech technology is gaining increasing attention in recent years. It provides an interactive platform for students to learn foreign languages. It is an objective of our work to develop such a system to help students to learn English sentence stress. This paper presents a novel method based on Hidden Markov Model (HMM) to detect English sentence stress. Different features, including Mel-Frequency Cepstrum Coefficients (MFCCs), energy and pitch, are investigated to detect English sentence stress and the results are then compared. The ability of each dimension of an individual feature to differentiate stressed vowels is evaluated. The best result, 80.6%, is obtained by combining normalized pitch values and MFCCs. And our method has the advantages of speaker independent and text independent.
Automatic lexical stress and pitch accent detection for L2 English speech using multi-distribution deep neural networks

Kun Li,Shaoguang Mao,Xu Li,Zhiyong Wu,Helen Meng

DOI: https://doi.org/10.1016/j.specom.2017.11.003

IF: 2.723

2018-02-01

Speech Communication

Abstract:This paper investigates the use of multi-distribution deep neural networks (MD-DNNs) for automatic lexical stress detection and pitch accent detection, which are useful for suprasegmental mispronunciation detection and diagnosis in second-language (L2) English speech. The features used in this paper cover syllable-based prosodic features (including maximum syllable loudness, syllable nucleus duration and a pair of dynamic pitch values) as well as lexical and syntactic features (encoded as binary variables). As stressed/accented syllables are more prominent than their neighbors, the two preceding and two following syllables are also taken into consideration. Experimental results show that the MD-DNN for lexical stress detection achieves an accuracy of 87.9% in syllable classification (primary/secondary/no stress) for words with three or more syllables. This performance is much better than those of our previous work using Gaussian mixture models (GMMs) and the prominence model (PM), whose accuracies are 72.1% and 76.3% respectively. Approached similarly as the lexical stress detector, the pitch accent detector obtains an accuracy of 90.2%, which is better than the results of using the GMMs and PM by about 9.6% and 6.9% respectively.

computer science, interdisciplinary applications,acoustics
Spoken English Assessment System for Non-Native Speakers Using Acoustic and Prosodic Features.

Qin Shi,Kun Li,ShiLei Zhang,Stephen M. Chu,Ji Xiao,ZhiJian Ou

DOI: https://doi.org/10.21437/interspeech.2010-541

2010-01-01

Abstract:The absence of real-time and targeted feedback is often critical in spoken foreign language learning. Computer-assisted language assessment systems are playing an ever more important role in this domain. This work considers the idiosyncratic pronunciation patterns of Chinese English speakers and uses both acoustic and prosody features to capture pronunciation, word stress, and rhythm information. The proposed system uses a. automatic speech recognition and alignment for pronunciation assessment, b. a set of special features with appropriate normalization for word stress detection, and c. a prosody phrase prediction model for rhythm assessment; and is shown to give immediate and accurate analyses to speakers to improve learning efficiency.
Study on Automatic Prediction of Sentential Stress with Natural Style in Chinese

SHAO Yanqiu,HAN Jiqing,LIU Ting,ZHAO Yongzhen

DOI: https://doi.org/10.3321/j.issn:0371-0025.2006.03.003

2006-01-01

Acta Acustica

Abstract:Stress is an important parameter for prosody processing in speech synthesis. This paper compares the acoustic features of unstressed syllable and stressed syllables with normally stressed syllables, including pitch, syllable duration, intensity and pause length. The relations between duration and pitch, and Third Tone (T3) and pitch are also studied. Three stress prediction models based on ANN, i.e. acoustic model, linguistic model and mixed model, are proposed to predict Chinese sentential stress. The result shows that mixed model is better than the other two models. In order to solve the problem of the diversity of manual labeling, another valuation method of support ratio is proposed.
Automatic Pitch Accent Detection Using Auto-Context with Acoustic Features.

Junhong Zhao,Wei-Qiang Zhang,Hua Yuan,Jia Liu,Shanhong Xia

DOI: https://doi.org/10.1109/iscslp.2012.6423523

2012-01-01

Abstract:In prosody event detection field, many local acoustic features have been proposed for representing the prosody characteristics of speech unit. The context information that represents some possible regularities underlying neighboring prosody events, however, hasn't been used effectively. The main difficulty to utilize prosodic context is that it's hard to capture the long-distance sequential dependency. In order to solve this problem, we introduce a new learning approach: auto-context. In this algorithm, a classifier is first trained based on local acoustic features; the discriminative probabilities produced by the classifier are selected as context information for the next iteration. Then a new classifier is trained by using the selected context information and local acoustic features. Repeating using the updated probabilities as the context information for the next iteration, the algorithm can boost recognition ability during its iterative process until converged. The merit of this method is that it can choose context information in a flexible way, while reserving reliable context information and abandoning unreliable ones. The experimental results showed that the proposed method improved the accuracy by absolutely about 1% for pitch accent detection.
Study on Automatic Prediction of Sentential Stress for Chinese Putonghua Text-to-Speech System with Natural Style

Ting Liu

2007-01-01

Abstract:Stress is an important parameter for prosody processing in speech synthesis. In this paper, we compare the acoustic features of neutral tone syllables and strong stress syllables with moderate stress syllables, including pitch, syllable duration, intensity and pause length after syllable. The relation between duration and pitch, as well as the Third Tone (T3) and pitch are also studied. Three stress prediction models based on ANN, i.e. the acoustic model, the linguistic model and the mixed model, are presented for predicting Chinese sentential stress. The results show that the mixed model performs better than the other two models. In order to solve the problem of the diversity of manual labeling, an evaluation index of support ratio is proposed.
Study on Stress Perception in Chinese Speech

Wei-xiang HU,Hong-hui DONG,Jian-hua TAO,Tai-yi HUANG

DOI: https://doi.org/10.3969/j.issn.1003-0077.2005.06.012

2005-01-01

Abstract:Restricted by prosody hierarchy and disturbed by tone and intonation,it is a hard task to detect the stress of Chinese speech automatically.In this paper, aiming at automatic stress perception in normal mandarin reading speech,we studied some acoustical measurements based on F0,duration and intensity and proposed a novel model to calculate the stress of each syllable.With a structure of classify tree,the model combined the restriction of tone context and prosody hierarchy effectively.It was shown from the result that the top line of pitch,pitch range,duration are important cues for stress perception.The model we developed can detect 80% accent syllable from corpus.
Automatic spoken English test for Chinese learners

Weiqian Liang,Jia Liu,RunSheng Liu

DOI: https://doi.org/10.1109/ICCCAS.2005.1495244

2005-01-01

Abstract:In this paper, we present research towards developing an automatic spoken language test for Chinese learners of English-as-a-foreign-language. The system can record and process learners' speech and instantly provide segmental and prosodic scores on pronunciation quality. To achieve high correlation between human and automatic scores, we employed a hidden Markov model based speech recognizer to align speech with the expected answer. Afterwards, segmental feature was assessed as a confidence score at the phoneme level. The prosodic aspects of the pronunciation were examined by using the detected duration, energy and pitch cues. Experiments were performed on a Mandarin Chinese accent speech corpus which was designed and collected by the authors. Results show that these automatic scores enabled reliable estimation of human scores. In particular, we achieved a 0.90 correlation with human scores for the segmental evaluation at the speaker level.
Improving l2 english rhythm evaluation with automatic sentence stress detection

Binghuai Lin,Liyuan Wang,Hongwei Ding,Xiaoli Feng

DOI: https://doi.org/10.1109/SLT48900.2021.9383455

2021-01-01

Abstract:English is a stress-timed language, for which sentence stress or prosodic stress plays an important role. It's then difficult for Chinese who are used to the syllable-timed rhythm to learn the rhythm of English [1]. In this paper, we investigate how to improve the rhythm evaluation based on the sentence stress for Chinese who learn English as a second language (ESL). Particularly, we explore some rhythm measures to quantify rhythmic differences among second language (L2) learners based on sentence stress. To relieve the dependency on labeled data of sentence stress, we predict sentence stress automatically utilizing a hierarchical network with bidirectional Long Short-Term Memory (BLSTM) [2]. We evaluate the proposed method based on the corpus consisting of 3,500 sentences recorded by 100 Chinese speakers aging from 10 to 20 years old, which was marked with the sentence stress labels and scored by three experts. Experimental results show the proposed sentence stress measure is well correlated with labeled prosody scores with a correlation coefficient of -0.73 and the automatic labeling method achieves comparable results with the method with gold labels.
Stress Accent Detection in an English Learning System

Runsheng Liu

2008-01-01

Abstract:The detector plays an important part in stress accent detection. This paper focuses on the development of a stress detector for an automatic system capable of identifying prominent syllables in utterances. The method combines multiple classifiers. Each classifier uses different Gaussian mixture models (GMM). The combination of multiple classifiers depends on each individual classifier's decision information from both training and testing samples. The results show that the combination of several discriminators is better than individual classifiers and achieves higher recognition rates and better reliability, especially with a small number of mixtures. Thus, this method can effectively detect accent stresses.
Detection of Lexical Stress Errors in Non-Native (L2) English with Data Augmentation and Attention

Daniel Korzekwa,Roberto Barra-Chicote,Szymon Zaporowski,Grzegorz Beringer,Jaime Lorenzo-Trueba,Alicja Serafinowicz,Jasha Droppo,Thomas Drugman,Bozena Kostek

DOI: https://doi.org/10.48550/arXiv.2012.14788

2021-06-07

Abstract:This paper describes two novel complementary techniques that improve the detection of lexical stress errors in non-native (L2) English speech: attention-based feature extraction and data augmentation based on Neural Text-To-Speech (TTS). In a classical approach, audio features are usually extracted from fixed regions of speech such as the syllable nucleus. We propose an attention-based deep learning model that automatically derives optimal syllable-level representation from frame-level and phoneme-level audio features. Training this model is challenging because of the limited amount of incorrect stress patterns. To solve this problem, we propose to augment the training set with incorrectly stressed words generated with Neural TTS. Combining both techniques achieves 94.8% precision and 49.2% recall for the detection of incorrectly stressed words in L2 English speech of Slavic and Baltic speakers.

Audio and Speech Processing,Sound
Deep-Learning-Based Automated Classification of Chinese Speech Sound Disorders

Yao-Ming Kuo,Shanq-Jang Ruan,Yu-Chin Chen,Ya-Wen Tu

DOI: https://doi.org/10.3390/children9070996

2022-07-01

Children

Abstract:This article describes a system for analyzing acoustic data to assist in the diagnosis and classification of children’s speech sound disorders (SSDs) using a computer. The analysis concentrated on identifying and categorizing four distinct types of Chinese SSDs. The study collected and generated a speech corpus containing 2540 stopping, backing, final consonant deletion process (FCDP), and affrication samples from 90 children aged 3–6 years with normal or pathological articulatory features. Each recording was accompanied by a detailed diagnostic annotation by two speech–language pathologists (SLPs). Classification of the speech samples was accomplished using three well-established neural network models for image classification. The feature maps were created using three sets of MFCC (Mel-frequency cepstral coefficients) parameters extracted from speech sounds and aggregated into a three-dimensional data structure as model input. We employed six techniques for data augmentation to augment the available dataset while avoiding overfitting. The experiments examine the usability of four different categories of Chinese phrases and characters. Experiments with different data subsets demonstrate the system’s ability to accurately detect the analyzed pronunciation disorders. The best multi-class classification using a single Chinese phrase achieves an accuracy of 74.4 percent.

pediatrics
Automatic Stress Exaggeration By Prosody Modification To Assist Language Learners Perceive Sentence Stress

Jingli Lu,Ruili Wang,Liyanage C. De Silva

DOI: https://doi.org/10.1007/s10772-011-9124-2

2012-01-01

International Journal of Speech Technology

Abstract:In this paper, we propose a set of automatic stress exaggeration methods that can enlarge the differences between stressed and unstressed syllables. Our stress exaggeration methods can be used in computer-aided language learning systems to assist second language learners perceive stress patterns. The intention of our automatic stress exaggeration methods is to support hyper-pronunciation training which is commonly used in classrooms by teachers. In hyper-pronunciation training, exaggeration is used to help learners increase their awareness of acoustic features and effectively apply these features into their pronunciation. Duration, pitch and intensity have been claimed to be the main acoustic features that are closely related to stress in English language. Thus, four stress exaggeration methods are proposed in this paper: (i) duration-based stress exaggeration, (ii) pitch-based stress exaggeration, (iii) intensitybased stress exaggeration, and (iv) a combined stress exaggeration method that integrates the duration-based, pitchbased and intensity-based exaggeration methods. Our perceptual experimental results show that resynthesised stimuli by our proposed stress exaggerated methods can help learners of English as a Second Language (ESL) better perceive English stress patterns significantly.
The Transfer of L1 Acoustic Cues in the Perception of L2 Lexical Stress

Qian Wang,Tae-Jin Yoon

2008-01-01

Abstract:A study was conducted to investigate the use of acoustic cues by Chinese learners of English in stress perception. The study included the stress judgment of synthesized disyllabic nonsense tokens and an oddity test using real English words from carrier sentences. The study also manipulated F0 difference on disyllabic nonsense words in 5 level, while the duration and intensity cues remained consistent on the syllables. Disyllabic words like Latmab, nizdit, and tetsep were developed during the study. The study also had two groups of participants to listen to the tokens and an experimental group of 68 CE. It was observed that the difference between CE and NE in stress perception caused by their reliance on acoustic cues. This experiment gave same results with Vietnamese learners and Japanese learners.
Automatic Mispronunciation Detection for Mandarin Chinese

ZHANG Feng,HUANG Chao,DAI Lirong

DOI: https://doi.org/10.3969/j.issn.1003-0077.2010.02.015

2010-01-01

Abstract:The current automatic mispronunciation detection systems are mostly based on automatic speech recognition(ASR) framework with statistical model.This paper presents the methods to improve the performance of mispronunciation detection at syllable level for Mandarin Chinese from two aspects: introducing the speaker adaptive training(SAT) and the selective maximum likelihood linear regression(SMLLR) to get a better acoustic statistical model,and proposing speaker normalization backend because of the limited information and the different rating level for the different pronunciation level.Experiments on a database of 8000 syllables pronounced by 40 speakers with varied pronunciation proficiency indicate the promising effects of these strategies by improving the precision from 45.8% to 53.6% at 30% recall,and 64.6% to 79.9% at 10% recall.
Automatic detection of contrastive word pairs using textual and acoustic features

Zang Xiao,Wu Zhiyong,Ning Yishuang,Meng Helen,Cai Lianhong

DOI: https://doi.org/10.1109/ICOSP.2014.7015073

IF: 4.729

2014-01-01

Signal Processing

Abstract:Labeling emphatic words from speech recordings plays an important role in building speech corpus for expressive speech synthesis. People generally pronounce some words stronger than usual, making the speech more expressive and signaling the focus of the sentence. Contrastive word pairs are often pronounced with stronger prominences and their presence modifies the meaning of the utterance in subtle but important ways. We used a subset of Switchboard corpus to study the acoustic characteristics of contrastive word pairs and the differences between contrastive and non-contrastive words. To address the problem of automatic detection of contrastive word pairs, support vector machines (SVMs) are used to automatically detect contrastive word pairs. We report the results for automatic detection of contrastive word pairs based on textual and acoustic features. By adding acoustic features, a much better performance is achieved.
Stress Predicition for Mandarin Text-to-speech System Using Discourse Context Feature.

Hao Che,Jianhua Tao

DOI: https://doi.org/10.1109/icsda.2013.6709908

2013-01-01

Abstract:Stress prediction is a vital factor for both speech synthesis and natural speech understanding. In this paper, we investigate how to improve the performance of Mandarin stress predictor by introducing discourse context features. Two widely accepted statistical methods are employed to evaluate given/new status and informativeness of a word which are two major discourse context features in the stress prediction. Syntactic and other sentence context features are used in the method as well. The machine learning algorithm, Maximum Entropy model, is adopted to predict which syllable will be stressed with the textual features. The experimental result shows that the performance of the predictor with the discourse context features is better than the methods using the sentence features and even close to the methods using both the acoustic and textual features.
Daily stress detection from real-life speeches using acoustic and semantic information

Peixian Lu,Liuxing Tsao,Liang Ma

DOI: https://doi.org/10.1080/00140139.2024.2430370

IF: 2.5612

2024-11-28

Ergonomics

Abstract:Detecting daily stress is of vital importance for workplace safety and health, and natural speech is recommended as one of the main methods of mental stress detection. This study developed machine-learning models for daily stress detection from real-life speeches by fusing its acoustic and semantic signals. First, we collected real-life speech data from life-stress-catharsis room of online chat platform and established a speech database with real daily stress. Second, we obtained the model performances of common machine-learning classifiers for stress detection and compared them with human performance. The stress-detection classifiers achieved a promising performance of 74.25% accuracy and 83.73% F1-score using only acoustic signal. By fusing with the semantic signal, the stress detection model performance was significantly improved and achieved a performance of 81.20% accuracy and 87.46% F1-score, which validated the importance of semantic information in daily stress detection. Meanwhile, the best performance of the machine learning model was close to the human recognition capability. The results of this study validated the feasibility of detecting daily stress based on real speech. The models developed in this study could be used for daily stress detection in real life and can provide information for stress interventions to ease the negative effects on health. This study developed machine-learning models for daily stress detection from real-life speeches by fusing its acoustic and semantic signals. The results validated the feasibility of detecting daily stress based on real speech, and the importance of semantic information in daily stress detection. The model developed in this study could be used for daily stress detection in real life.

engineering, industrial,ergonomics,psychology, applied

Automatic lexical stress detection using acoustic features for computer-assisted language learning

Automatic Lexical Stress Detection for English Learning

AUTOMATIC STRESS PREDICTION OF CHINESE SPEECH SYNTHESIS

English Sentence Stress Detection System Based on HMM Framework

Automatic lexical stress and pitch accent detection for L2 English speech using multi-distribution deep neural networks

Spoken English Assessment System for Non-Native Speakers Using Acoustic and Prosodic Features.

Study on Automatic Prediction of Sentential Stress with Natural Style in Chinese

Automatic Pitch Accent Detection Using Auto-Context with Acoustic Features.

Study on Automatic Prediction of Sentential Stress for Chinese Putonghua Text-to-Speech System with Natural Style

Study on Stress Perception in Chinese Speech

Automatic spoken English test for Chinese learners

Improving l2 english rhythm evaluation with automatic sentence stress detection

Stress Accent Detection in an English Learning System

Detection of Lexical Stress Errors in Non-Native (L2) English with Data Augmentation and Attention

Deep-Learning-Based Automated Classification of Chinese Speech Sound Disorders

Automatic Stress Exaggeration By Prosody Modification To Assist Language Learners Perceive Sentence Stress

The Transfer of L1 Acoustic Cues in the Perception of L2 Lexical Stress

Automatic Mispronunciation Detection for Mandarin Chinese

Automatic detection of contrastive word pairs using textual and acoustic features

Stress Predicition for Mandarin Text-to-speech System Using Discourse Context Feature.

Daily stress detection from real-life speeches using acoustic and semantic information