Abstract:Long short-term memory (LSTM) has proven effective in modeling sequential data. However, it may encounter challenges in accurately capturing long-term temporal dependencies. LSTM plays a central role in speech enhancement by effectively modeling and capturing temporal dependencies in speech signals. This paper introduces a variable-neurons-based LSTM designed for capturing long-term temporal dependencies by reducing neuron representation in layers with no loss of data. A skip connection between nonadjacent layers is added to prevent gradient vanishing. An attention mechanism in these connections highlights important features and spectral components. Our LSTM is inherently causal, making it well-suited for real-time processing without relying on future information. Training involves utilizing combined acoustic feature sets for improved performance, and the models estimate two time–frequency masks—the ideal ratio mask (IRM) and the ideal binary mask (IBM). Comprehensive evaluation using perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI) showed that the proposed LSTM architecture demonstrates enhanced speech intelligibility and perceptual quality. Composite measures further substantiated performance, considering residual noise distortion (Cbak) and speech distortion (Csig). The proposed model showed a 16.21% improvement in STOI and a 0.69 improvement in PESQ on the TIMIT database. Similarly, with the LibriSpeech database, the STOI and PESQ showed improvements of 16.41% and 0.71 over noisy mixtures. The proposed LSTM architecture outperforms deep neural networks (DNNs) in different stationary and nonstationary background noisy conditions. To train an automatic speech recognition (ASR) system on enhanced speech, the Kaldi toolkit is used for evaluating word error rate (WER). The proposed LSTM at the front-end notably reduced WERs, achieving a notable 15.13% WER across different noisy backgrounds.

Dynamic temporal residual network for sequence modeling

Residual Recurrent Neural Networks for Learning Sequential Representations.

Dependency-based Siamese Long Short-Term Memory Network for Learning Sentence Representations.

Dynamic temporal residual learning for speech recognition

Deep Differential Recurrent Neural Networks

Residual Memory Networks: Feed-forward approach to learn long temporal dependencies

Residual LSTM: Design of a Deep Recurrent Architecture for Distant Speech Recognition

On extended long short-term memory and dependent bidirectional recurrent neural network

Towards Efficient Recurrent Architectures: A Deep LSTM Neural Network Applied to Speech Enhancement and Recognition

Memory Visualization for Gated Recurrent Neural Networks in Speech Recognition

Delayed Memory Unit: Modelling Temporal Dependency Through Delay Gate

Sequence Classification Restricted Boltzmann Machines With Gated Units

Differential Recurrent Neural Network and its Application for Human Activity Recognition

PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning

Long Short-Term Memory with Dynamic Skip Connections.

Hierarchically Gated Recurrent Neural Network for Sequence Modeling

ELSTM: An improved long short‐term memory network language model for sequence learning

Gated Sequential Recommendation with Dynamic Memory Network

TS-LSTM and Temporal-Inception: Exploiting Spatiotemporal Dynamics for Activity Recognition

Tree Memory Networks for Modelling Long-term Temporal Dependencies

Residual Recurrent Neural Network with Sparse Training for Offline Arabic Handwriting Recognition