Abstract:Long short-term memory (LSTM) has proven effective in modeling sequential data. However, it may encounter challenges in accurately capturing long-term temporal dependencies. LSTM plays a central role in speech enhancement by effectively modeling and capturing temporal dependencies in speech signals. This paper introduces a variable-neurons-based LSTM designed for capturing long-term temporal dependencies by reducing neuron representation in layers with no loss of data. A skip connection between nonadjacent layers is added to prevent gradient vanishing. An attention mechanism in these connections highlights important features and spectral components. Our LSTM is inherently causal, making it well-suited for real-time processing without relying on future information. Training involves utilizing combined acoustic feature sets for improved performance, and the models estimate two time–frequency masks—the ideal ratio mask (IRM) and the ideal binary mask (IBM). Comprehensive evaluation using perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI) showed that the proposed LSTM architecture demonstrates enhanced speech intelligibility and perceptual quality. Composite measures further substantiated performance, considering residual noise distortion (Cbak) and speech distortion (Csig). The proposed model showed a 16.21% improvement in STOI and a 0.69 improvement in PESQ on the TIMIT database. Similarly, with the LibriSpeech database, the STOI and PESQ showed improvements of 16.41% and 0.71 over noisy mixtures. The proposed LSTM architecture outperforms deep neural networks (DNNs) in different stationary and nonstationary background noisy conditions. To train an automatic speech recognition (ASR) system on enhanced speech, the Kaldi toolkit is used for evaluating word error rate (WER). The proposed LSTM at the front-end notably reduced WERs, achieving a notable 15.13% WER across different noisy backgrounds.

Speech recognition by using the Extended Associative Memory Neural Network (EAMNN)

Neural network noisy speech recognition and understanding with information feedback

A robust learning algorithm for the extended AM neural network

Research on the Model of Speech Recognition and Understanding by Using Hierarchical Information Feedback

Speech Recognition by Extended Loop Neural Network

Embedding Encoder-Decoder with Attention Mechanism for Monaural Speech Enhancement

Speech-enhanced and Noise-aware Networks for Robust Speech Recognition

A Noise-Aware Memory-Attention Network Architecture for Regression-Based Speech Enhancement.

Environment-Dependent Attention-Driven Recurrent Convolutional Neural Network for Robust Speech Enhancement

Adaptive-U-net: Conditional Neural Network for Monaural Speech Enhancement using Memory Mechanism

Adaptive Very Deep Convolutional Residual Network for Noise Robust Speech Recognition

An efficient joint training model for monaural noisy-reverberant speech recognition

Towards Efficient Recurrent Architectures: A Deep LSTM Neural Network Applied to Speech Enhancement and Recognition

NeuralEcho: A Self-Attentive Recurrent Neural Network For Unified Acoustic Echo Suppression And Speech Enhancement

Deep causal speech enhancement and recognition using efficient long-short term memory Recurrent Neural Network

Multi-task Joint-Learning of Deep Neural Networks for Robust Speech Recognition

Neural Network Based Multi-Factor Aware Joint Training for Robust Speech Recognition.

Optimum associative neural network utilizing maximum likelihood

Improving Accented Mandarin Speech Recognition by Using Recurrent Neural Network Based Language Model Adaptation

Application Research of Speech Emotion Recognition Based on Elnan Neural Network

Speech Enhancement Method Based on LSTM Neural Network for Speech Recognition