Abstract:Phonological-based features (articulatory features, AFs) describe the movements of the vocal organ which are shared across languages. This paper investigates a domain-adversarial neural network (DANN) to extract reliable AFs, and different multi-stream techniques are used for cross-lingual speech recognition. First, a novel universal phonological attributes definition is proposed for Mandarin, English, German and French. Then a DANN-based AFs detector is trained using source languages (English, German and French). When doing the cross-lingual speech recognition, the AFs detectors are used to transfer the phonological knowledge from source languages (English, German and French) to the target language (Mandarin). Two multi-stream approaches are introduced to fuse the acoustic features and cross-lingual AFs. In addition, the monolingual AFs system (i.e., the AFs are directly extracted from the target language) is also investigated. Experiments show that the performance of the AFs detector can be improved by using convolutional neural networks (CNN) with a domain-adversarial learning method. The multi-head attention (MHA) based multi-stream can reach the best performance compared to the baseline, cross-lingual adaptation approach, and other approaches. More specifically, the MHA-mode with cross-lingual AFs yields significant improvements over monolingual AFs with the restriction of training data size and, which can be easily extended to other low-resource languages.

Cross-language speech attribute detection and phone recognition for Tibetan using deep learning

Shared Speech Attribute Augmentation for English-Tibetan Cross-Language Phone Recognition

Phone Recognition for Lhasa-Tibetan Based on Articulatory Features Augmentation Learning

Multilingual Articulatory Features Augmentation Learning

Cross-language Transfer Speech Recognition Using Deep Learning.

Improving Minority Language Speech Recognition Based on Distinctive Features

Deep Feature Learning For Tibetan Speech Recognition Using Sparse Auto-Encoder

Speech Recognition Based on Deep Neural Networks on Tibetan Corpus

Cross-lingual Automatic Speech Recognition Exploiting Articulatory Features

Detection-based accented speech recognition using articulatory features.

Comparison of Two Cross-lingual AF Extraction Methods

Cross-Language Transfer Learning-based Lhasa-Tibetan Speech Recognition

Investigation on Acoustic Modeling with Different Phoneme Set for Continuous Lhasa Tibetan Recognition Based on Dnn Method

Audio-Visual Tibetan Speech Recognition Based On A Deep Dynamic Bayesian Network For Natural Human Robot Interaction Regular Paper

Selection of acoustic modeling unit for Tibetan speech recognition based on deep learning

Domain-Adversarial Based Model with Phonological Knowledge for Cross-Lingual Speech Recognition

Tibetan Multi-dialect Speech and Dialect Identity Recognition

Lhasa Dialect Recognition of Different Phonemes Based on TDNN Method.

Articulatory Feature Based Multilingual MLPs for Low-Resource Speech Recognition.

International Journal of Advanced Robotic Systems Audio-Visual Tibetan Speech Recognition Based on a Deep Dynamic Bayesian Network for Natural Human Robot Interaction Regular Paper

Tibetan Multi-Dialect Speech Recognition Using Latent Regression Bayesian Network and End-To-End Mode