Abstract:BACKGROUND AND OBJECTIVE: Most dysarthric patients encounter communication problems due to unintelligible speech. Currently, there are many voice-driven systems aimed at improving their speech intelligibility; however, the intelligibility performance of these systems are affected by challenging application conditions (e.g., time variance of patient's speech and background noise). To alleviate these problems, we proposed a dysarthria voice conversion (DVC) system for dysarthric patients and investigated the benefits under challenging application conditions.METHOD: A deep learning-based voice conversion system with phonetic posteriorgram (PPG) features, called the DVC-PPG system, was proposed in this study. An objective-evaluation metric of Google automatic speech recognition (Google ASR) system and a listening test were used to demonstrate the speech intelligibility benefits of DVC-PPG under quiet and noisy test conditions; besides, the well-known voice conversion system using mel-spectrogram, DVC-Mels, was used for comparison to verify the benefits of the proposed DVC-PPG system.RESULTS: The objective-evaluation metric of Google ASR showed the average accuracy of two subjects in the duplicate and outside test conditions while the DVC-PPG system provided higher speech recognitions rate (83.2% and 67.5%) than dysarthric speech (36.5% and 26.9%) and DVC-Mels (52.9% and 33.8%) under quiet conditions. However, the DVC-PPG system provided more stable performance than the DVC-Mels under noisy test conditions. In addition, the results of the listening test showed that the speech-intelligibility performance of DVC-PPG was better than those obtained via the dysarthria speech and DVC-Mels under the duplicate and outside conditions, respectively.CONCLUSIONS: The objective-evaluation metric and listening test results showed that the recognition rate of the proposed DVC-PPG system was significantly higher than those obtained via the original dysarthric speech and DVC-Mels system. Therefore, it can be inferred from our study that the DVC-PPG system can improve the ability of dysarthric patients to communicate with people under challenging application conditions.

Research on the algorithm of consonant feature extraction for speech training aid

A Novel and Efficient Voice Activity Detector Using Shape Features of Speech Wave.

Pronunciation recognition of English phonemes /\textipa{@}/, /æ/, /\textipa{A}:/ and /\textipa{2}/ using Formants and Mel Frequency Cepstral Coefficients

Utilizing auxiliary data in phoneme recognition based on Articulatory Feature

Leveraging Laryngograph Data for Robust Voicing Detection in Speech

Language-Independent Approach for Automatic Computation of Vowel Articulation Features in Dysarthric Speech Assessment

A High-Performance Auditory Feature for Robust Speech Recognition

A Speech Endpoint Detection Method Based on Consonance Energy

Effective Acoustic Modeling for Pronunciation Quality Scoring of Strongly Accented Mandarin Speech

An Auditory System-Based Feature for Robust Speech Recognition.

Objective Evaluation Of Glottal Stops In Cleft Palate Speech Based On Spectral Moments

A novel speech processing algorithm for cochlear implant based on selective fundamental frequency control

Design and implementation of speech recognition algorithm based on frequency range

Detection-based accented speech recognition using articulatory features.

Articulatory-to-Acoustic Conversion with Cascaded Prediction of Spectral and Excitation Features Using Neural Networks

Evaluation of Benefits from a Computer-Aided Pronunciation Training System for German Learners of Mandarin Chinese

Articulatory-to-acoustic Conversion Using BLSTM-RNNs with Augmented Input Representation.

Phonetic posteriorgram-based voice conversion system to improve speech intelligibility of dysarthric patients

Automatic Detection System for Velopharyngeal Insufficiency Based on Acoustic Signals from Nasal and Oral Channels

Mapping the spectrotemporal regions influencing perception of French stop consonants in noise

Application of the Artificial Intelligence Algorithm in the Automatic Segmentation of Mandarin Dialect Accent