Roles of Computational Auditory Model in Automatic Speech Recognition

Hs Chi,Xh Wu

2000-01-01

Progress in Natural Science

Abstract:The development of computational auditory modeling in the last 30 years is reviewed, and its development in the coming years is predicted.

What problem does this paper attempt to address?

Progress in computational modeling of auditory periphery

LI Zhaohui,CHI Huisheng

DOI: https://doi.org/10.3321/j.issn:0371-0025.2006.05.010

2006-01-01

Abstract:After introducing the characteristics of auditory periphery, the progress in the computational modeling of auditory periphery within last three decades is reviewed, and several suggestions are made for the modeling in the speech applications. Then, its possibility in passive sonar target classification is discussed, and some proposals are given for the adaptation of the modeling, according to the specialties of sonar processing.
Wavoice: A mmWave-assisted Noise-resistant Speech Recognition SystemJust Accepted

Tiantian Liu,Chao Wang,Zhengxiong Li,Ming-Chun Huang,Wenyao Xu,Feng Lin

DOI: https://doi.org/10.1145/3597457

2022-01-01

ACM Transactions on Sensor Networks

Abstract:As automatic speech recognition evolves, the deployment of voice user interface has boomingly expanded. Especially since the COVID-19 pandemic, VUI has gained more attention in online communication owing to its non-contact property. However, VUI struggles to be applied in public scenes due to the degradation of received audio signals caused by various ambient noises. In this paper, we propose Wavoice , the first noise-resistant multi-modal speech recognition system that fuses two distinct voice sensing modalities, i.e., millimeter-wave (mmWave) signals and audio signals from a microphone, together. One key contribution is to model the inherent correlation between mmWave and audio signals. Based on it, Wavoice facilitates the real-time noise-resistant voice activity detection and user targeting from multiple speakers. Additionally, we elaborate on two novel modules for multi-modal fusion embedded into the neural network, leading to accurate speech recognition. Extensive experiments prove the effectiveness of Wavoice under adverse conditions, that is, the character recognition error rate below 1 \(\% \) in a range of 7 meters. In terms of robustness and accuracy, Wavoice considerably outperforms existing audio-only speech recognition methods with lower character error rate and word error rate.
Modeling of Three Types of Auditory Nerve and Its Application in Speech Recognition.

Zhimin Liu,Xihong Wu,Bin Zhen,Huisheng Chi

2000-01-01

Abstract:A novel auditory nerve model is described here which simulates the three types of auditory nerves existing in the auditory system. The inspiration of the model is the absence of the simulation of the different types of auditory nerves in current auditory models. Based on the previous work, three sub-models replace the prevailing single auditory nerve discharge model in the common peripheral auditory models. Three auditory features were extracted and applied in speech recognition experiments. The results show that three models have different noise-resistant properties and the model with large dynamic range is exceptionally robust in speech recognition.
Automatic Speech Recognition : A Study and Performance Evaluation on Neural Networks and Hidden Markov Models

Antonio G. Thom

Abstract:The main goal in this research is to find out possible ways to built hybrid systems, based on neural network (NN) and hidden Markov (HMM) models, for the task of automatic speech recognition. The investigation that has been conducted covers different types of neural network and hidden Markov models, and the combination of them into some hybrid models. The neural networks used were basically MLP and Radial Basis models. The hidden Markov models were basically different combinations of states and mixtures of the Continuous Density type of the Bakis model. A reduced set with ten words spoken in the Portuguese idiom, from Brazil, was carefully chosen to provide some pronounce and phonetic confusion. The results already obtained showed very positive, pointing toward to a high potentiality of such hybrid models.

Computer Science
Development status, problems and solutions of speech recognition technology

Shuang Zhang

DOI: https://doi.org/10.1088/1742-6596/1693/1/012137

2020-12-01

Journal of Physics: Conference Series

Abstract:Abstract The purpose of automatic speech recognition technology is to enable the machine to “understand” human voice, and convert human voice into text information, which is the key technology to realize human-computer interaction. This paper briefly introduces the development of speech recognition technology and the acoustic model involved, analyzes the problems of noise interference in the environment, the low recognition rate of dialect speech, and the need to improve the speech recognition effect in the far-field environment, and puts forward solutions such as optimizing the acoustic model, building dialect corpus, accurately modeling the sound transmission environment, the future development is prospected.
A Forward Masking Auditory Model And Its Application In Speaker Identification And Speech Recognition

Zhimin Liu,Xihong Wu,Bin Zhen,Huisheng Chi

IF: 1.019

2001-01-01

Chinese Journal of Electronics

Abstract:A novel computational auditory model which simulates the forward-masking mechanism of auditory nerve discharge is presented. Both features based on the model are extracted: FMFRC (forward masking firing-rate cepstrum) and FMSRC (forward masking synchronized rate cepstrum). Isolated-word speech recognition and text-dependent speaker identification experiments based on TI46 are performed. The results show that the new features based on the forward masking model is far more robust than MFCC (mel-frequency cepstrum coefficients) and the performance will be improved compared to the features without such dynamic property. Moreover, the model and the feature extraction method based on it are feasible in practice and promising in robust speech recognition and speaker identification.
Computer Audition: From Task-Specific Machine Learning to Foundation Models

Andreas Triantafyllopoulos,Iosif Tsangko,Alexander Gebhard,Annamaria Mesaros,Tuomas Virtanen,Björn Schuller

2024-07-22

Abstract:Foundation models (FMs) are increasingly spearheading recent advances on a variety of tasks that fall under the purview of computer audition -- the use of machines to understand sounds. They feature several advantages over traditional pipelines: among others, the ability to consolidate multiple tasks in a single model, the option to leverage knowledge from other modalities, and the readily-available interaction with human users. Naturally, these promises have created substantial excitement in the audio community, and have led to a wave of early attempts to build new, general-purpose foundation models for audio. In the present contribution, we give an overview of computational audio analysis as it transitions from traditional pipelines towards auditory foundation models. Our work highlights the key operating principles that underpin those models, and showcases how they can accommodate multiple tasks that the audio community previously tackled separately.

Sound,Audio and Speech Processing
Space-and-speaker-aware Acoustic Modeling with Effective Data Augmentation for Recognition of Multi-Array Conversational Speech

Li Chai,Hang Chen,Jun Du,Qing-Feng Liu,Chin-Hui Lee

DOI: https://doi.org/10.1016/j.specom.2023.102958

IF: 2.723

2023-01-01

Speech Communication

Abstract:We propose a space-and-speaker-aware (SSA) approach to acoustic modeling (AM), denoted as SSA-AM, to improve system performances of automatic speech recognition (ASR) in distant multi-array conversational scenarios. In contrast to conventional AM which only uses spectral features from a target speaker as inputs, the inputs to SSA-AM consists of speech features from both the target and interfering speakers, which contain discriminative information from different speakers, including spatial information embedded in interaural phase differences (IPDs) between individual interfering speakers and the target speaker. In the proposed SSA-AM framework, we explore four acoustic model architectures consisting of different combinations of four neural networks, namely deep residual network, factorized time delay neural network, self-attention and residual bidirectional long short-term memory neural network. Various data augmentation techniques are adopted to expand the training data to include different options of beamformed speech obtained from multi-channel speech enhancement. Evaluated on the recent CHiME-6 Challenge Track 1, our proposed SSA-AM framework achieves consistent recognition performance improvements when compared with the official baseline acoustic models. Furthermore, SSA-AM outperforms acoustic models without explicitly using the space and speaker information. Finally, our data augmentation schemes are shown to be especially effective for compact model designs. Code is released at https://github.com/coalboss/SSA_AM.
A survey of technologies for automatic Dysarthric speech recognition

Zhaopeng Qian,Kejing Xiao,Chongchong Yu

DOI: https://doi.org/10.1186/s13636-023-00318-2

2023-11-11

Abstract:Abstract Speakers with dysarthria often struggle to accurately pronounce words and effectively communicate with others. Automatic speech recognition (ASR) is a powerful tool for extracting the content from speakers with dysarthria. However, the narrow concept of ASR typically only covers technologies that process acoustic modality signals. In this paper, we broaden the scope of this concept that the generalized concept of ASR for dysarthric speech. Our survey discussed the systems encompassed acoustic modality processing, articulatory movements processing and audio-visual modality fusion processing in the application of recognizing dysarthric speech. Contrary to previous surveys on dysarthric speech recognition, we have conducted a systematic review of the advancements in this field. In particular, we introduced state-of-the-art technologies to supplement the survey of recent research during the era of multi-modality fusion in dysarthric speech recognition. Our survey found that audio-visual fusion technologies perform better than traditional ASR technologies in the task of dysarthric speech recognition. However, training audio-visual fusion models requires more computing resources, and the available data corpus for dysarthric speech is limited. Despite these challenges, state-of-the-art technologies show promising potential for further improving the accuracy of dysarthric speech recognition in the future.

engineering, electrical & electronic,acoustics
Investigation of Acoustic Modeling Techniques for LVCSR Systems

X Liu,MJF Gales,KC Sim,K Yu

DOI: https://doi.org/10.1109/icassp.2005.1415247

2005-01-01

Abstract:The paper describes the use of several advanced acoustic modeling techniques for the 2004 CU-HTK large vocabulary speech recognition systems. These techniques include Gaussianization for speaker normalization, discriminative cluster adaptive training (CAT), subspace for precision and mean (SPAM) modeling of inverse covariances, and discriminative complexity control. Acoustic models featuring these techniques were integrated into a state-of-the-art 10 real-time multi-pass system with sophisticated adaptation for performance evaluation. Experimental results are presented on both broadcast news (BN) and conversational telephone speech (CTS) transcription tasks.
Context Dependent Syllable Acoustic Model For Continuous Chinese Speech Recognition

Hao Wu,Xihong Wu

DOI: https://doi.org/10.21437/interspeech.2007-481

2007-01-01

Abstract:The choice of basic modeling unit in building acoustic model for a continuous Mandarin speech recognition task is a very important issue [1]. Unlike traditional phoneme or Initial/Finals (IFs) units based acoustic modeling methods, which usually suffer from the limitations of less accuracy in modeling intrasyllable variations and long scale temporal dependencies, in this paper, a practicable syllable based approach is presented. In contrast with IFs, syllable can implicitly model the intrasyllable variations in good accuracy. Also, by carefully choosing context modeling schemes and parameter tying methods, syllable based acoustic model can capture longer temporal variations while keeping the complexity of model well controlled. Meanwhile, considering the data unbalanced problem, multiple sized unit model based approaches are also implemented in this research. The experiment result shows the acoustic model based on the presented syllable based approach is effective in improving the performance of the Chinese continuous speech recognition.
PHMM Based Asynchronous Acoustic Model for Chinese Large Vocabulary Continuous Speech Recognition

Hao Wu,Xihong Wu,Huisheng Chi

DOI: https://doi.org/10.1109/icassp.2009.4960624

2009-01-01

Abstract:In this paper, we presented an asynchronous multiple stream based Chinese tonal acoustic modeling framework. In this framework, toneless phonetic units and tones are modeled separately with different acoustic features. During the training and decoding process, a set of models are coupled together with a product hidden Markov models (PHMM) to represent whole tonal phonetic units. Through this, a compound context dependent tonal model can be generated from a few simple models. Experiments show that such model scheme generates more compact and accurate model presentation and brings improvement on the performance for large vocabulary speech recognition tasks.
Recent Progresses in Deep Learning based Acoustic Models (Updated)

Dong Yu,Jinyu Li

DOI: https://doi.org/10.48550/arXiv.1804.09298

2018-04-25

Audio and Speech Processing

Abstract:In this paper, we summarize recent progresses made in deep learning based acoustic models and the motivation and insights behind the surveyed techniques. We first discuss acoustic models that can effectively exploit variable-length contextual information, such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), and their various combination with other models. We then describe acoustic models that are optimized end-to-end with emphasis on feature representations learned jointly with rest of the system, the connectionist temporal classification (CTC) criterion, and the attention-based sequence-to-sequence model. We further illustrate robustness issues in speech recognition systems, and discuss acoustic model adaptation, speech enhancement and separation, and robust training strategies. We also cover modeling techniques that lead to more efficient decoding and discuss possible future directions in acoustic model research.
Context Dependent Initial/final Acoustic Modeling for Continuous Chinese Speech Recognition

李净,郑方,张继勇,吴文虎

DOI: https://doi.org/10.3321/j.issn:1000-0054.2004.01.016

2000-01-01

Abstract:Acoustic modeling is very important for continuous Chinese speech recognition. The extended Initial/Final (XIF) set chosen as the basic speech recognition unit set to analyze the Chinese language characteristics outperformed the standard IF set. Decision tree-based state tying technology was used to construct the context dependent Initial/Final acoustic model (Tri-XIF model), with an appropriate question set design based on Chinese linguistic knowledge. Methods were developed to optimize the Tri-XIF modeling, including transcription refinement, question set extension, and model size reduction. Tests show that the Tri-XIF modeling is much better than either Tri-phone modeling or syllable modeling, with the syllable error rate reduced by 24.53% relative to the Tri-phone model and 41.65% relative to syllable model. More than 20% model size reduction was obtained with little performance deterioration using the methods in the Tri-XIF model.
The Realization of Acoustic Model Based on Noise Modulating for Cochlear Implantation

Shengyang HUANG,Xiao BAI,Datian YE

DOI: https://doi.org/10.3969/j.issn.1002-3208.2006.06.004

2006-01-01

Abstract:Acoustic simulations attempt to evaluate speech processing strategies.Generally,there are two ways to set up a simulation model.One is based on modulated sine waves,and the other is generated through modulated noise.This article discussed about the latter type,which is also known as acoustic model.Here we took SPEAK strategy as an example and proposed its corresponding model.Then a designed acoustic simulation experiment was carried out.Results proved that this kind of acoustic simulation model was an effective and reasonable.This model provides a method to evaluate the speech processing strategies of cochlear implantation.
A model of speech recognition for hearing-impaired listeners based on deep learning

Jana Roßbach,Birger Kollmeier,Bernd T Meyer

DOI: https://doi.org/10.1121/10.0009411

Abstract:Automatic speech recognition (ASR) has made major progress based on deep machine learning, which motivated the use of deep neural networks (DNNs) as perception models and specifically to predict human speech recognition (HSR). This study investigates if a modeling approach based on a DNN that serves as phoneme classifier [Spille, Ewert, Kollmeier, and Meyer (2018). Comput. Speech Lang. 48, 51-66] can predict HSR for subjects with different degrees of hearing loss when listening to speech embedded in different complex noises. The eight noise signals range from simple stationary noise to a single competing talker and are added to matrix sentences, which are presented to 20 hearing-impaired (HI) listeners (categorized into three groups with different types of age-related hearing loss) to measure their speech recognition threshold (SRT), i.e., the signal-to-noise ratio with 50% word recognition rate. These are compared to responses obtained from the ASR-based model using degraded feature representations that take into account the individual hearing loss of the participants captured by a pure-tone audiogram. Additionally, SRTs obtained from eight normal-hearing (NH) listeners are analyzed. For NH subjects and three groups of HI listeners, the average SRT prediction error is below 2 dB, which is lower than the errors of the baseline models.
Toward a Better Understanding of Deep Neural Network Based Acoustic Modelling: An Empirical Investigation

Xingfu Wang,Lin Wang,Jing Chen,Litao Wu

DOI: https://doi.org/10.1609/aaai.v30i1.10256

2016-03-02

Proceedings of the AAAI Conference on Artificial Intelligence

Abstract:Recently, deep neural networks (DNNs) have outperformed traditional acoustic models on a variety of speech recognition benchmarks.However, due to system differences across research groups, although a tremendous breadth and depth of related work has been established, it is still not easy to assess the performance improvements of a particular architectural variant from examining the literature when building DNN acoustic models. Our work aims to uncover which variations among baseline systems are most relevant for automatic speech recognition (ASR) performance via a series of systematic tests on the limits of the major architectural choices.By holding all the other components fixed, we are able to explore the design and training decisions without being confounded by the other influencing factors. Our experiment results suggest that a relatively simple DNN architecture and optimization technique produces strong results.These findings, along with previous work, not only help build a better understanding towards why DNN acoustic models perform well or how they might be improved, but also help establish a set of best practices for new speech corpora and language understanding task variants.
Recent Advances in End-to-End Automatic Speech Recognition

Jinyu Li

DOI: https://doi.org/10.1561/116.00000050

2022-01-01

APSIPA Transactions on Signal and Information Processing

Abstract:Recently, the speech community is seeing a significant trend of moving from deep neural network based hybrid modeling to end-to-end (E2E) modeling for automatic speech recognition (ASR). While E2E models achieve the state-of-the-art results in most benchmarks in terms of ASR accuracy, hybrid models are still used in a large proportion of commercial ASR systems at the current time. There are lots of practical factors that affect the production model deployment decision. Traditional hybrid models, being optimized for production for decades, are usually good at these factors. Without providing excellent solutions to all these factors, it is hard for E2E models to be widely commercialized. In this paper, we will overview the recent advances in E2E models, focusing on technologies addressing those challenges from the industry’s perspective.
Exploring Sound Change Over Time: A Review of Computational and Human Perception

Siqi He,Wei Zhao

2024-07-06

Abstract:Computational and human perception are often considered separate approaches for studying sound changes over time; few works have touched on the intersection of both. To fill this research gap, we provide a pioneering review contrasting computational with human perception from the perspectives of methods and tasks. Overall, computational approaches rely on computer-driven models to perceive historical sound changes on etymological datasets, while human approaches use listener-driven models to perceive ongoing sound changes on recording corpora. Despite their differences, both approaches complement each other on phonetic and acoustic levels, showing the potential to achieve a more comprehensive perception of sound change. Moreover, we call for a comparative study on the datasets used by both approaches to investigate the influence of historical sound changes on ongoing changes. Lastly, we discuss the applications of sound change in computational linguistics, and point out that perceiving sound change alone is insufficient, as many processes of language change are complex, with entangled changes at syntactic, semantic, and phonetic levels.

Computation and Language
Deep neural networks for syllable based acoustic modeling in Chinese speech recognition.

Xiangang Li,Caifu Hong,Yuning Yang,Xihong Wu

DOI: https://doi.org/10.1109/APSIPA.2013.6694176

2013-01-01

Abstract:Recently, the deep neural networks (DNNs) based acoustic modeling methods have been successfully applied to many speech recognition tasks. This paper reports the work about applying DNNs for syllable based acoustic modeling in Chinese automatic speech recognition (ASR). Compared with initial/finals (IFs), syllable can implicitly model the intra-syllable variations in better accuracy. However, the context dependent syllable based modeling set holds too many units, bringing about heavy problems on modeling and decoding implementation. In this paper, a WFST decoding framework is applied. Moreover, the decision tree based state tying and DNNs based models are discussed for the acoustic model training. The experimental results show that compared with the traditional IFs based modeling method, the proposed syllable modeling method using DNNs is more robust for data sparsity problem, which indicates that it has the potential to obtain better performance for Chinese ASR.

Roles of Computational Auditory Model in Automatic Speech Recognition

Progress in computational modeling of auditory periphery

Wavoice: A mmWave-assisted Noise-resistant Speech Recognition SystemJust Accepted

Modeling of Three Types of Auditory Nerve and Its Application in Speech Recognition.

Automatic Speech Recognition : A Study and Performance Evaluation on Neural Networks and Hidden Markov Models

Development status, problems and solutions of speech recognition technology

A Forward Masking Auditory Model And Its Application In Speaker Identification And Speech Recognition

Computer Audition: From Task-Specific Machine Learning to Foundation Models

Space-and-speaker-aware Acoustic Modeling with Effective Data Augmentation for Recognition of Multi-Array Conversational Speech

A survey of technologies for automatic Dysarthric speech recognition

Investigation of Acoustic Modeling Techniques for LVCSR Systems

Context Dependent Syllable Acoustic Model For Continuous Chinese Speech Recognition

PHMM Based Asynchronous Acoustic Model for Chinese Large Vocabulary Continuous Speech Recognition

Recent Progresses in Deep Learning based Acoustic Models (Updated)

Context Dependent Initial/final Acoustic Modeling for Continuous Chinese Speech Recognition

The Realization of Acoustic Model Based on Noise Modulating for Cochlear Implantation

A model of speech recognition for hearing-impaired listeners based on deep learning

Toward a Better Understanding of Deep Neural Network Based Acoustic Modelling: An Empirical Investigation

Recent Advances in End-to-End Automatic Speech Recognition

Exploring Sound Change Over Time: A Review of Computational and Human Perception

Deep neural networks for syllable based acoustic modeling in Chinese speech recognition.