Abstract:Long short-term memory (LSTM) has proven effective in modeling sequential data. However, it may encounter challenges in accurately capturing long-term temporal dependencies. LSTM plays a central role in speech enhancement by effectively modeling and capturing temporal dependencies in speech signals. This paper introduces a variable-neurons-based LSTM designed for capturing long-term temporal dependencies by reducing neuron representation in layers with no loss of data. A skip connection between nonadjacent layers is added to prevent gradient vanishing. An attention mechanism in these connections highlights important features and spectral components. Our LSTM is inherently causal, making it well-suited for real-time processing without relying on future information. Training involves utilizing combined acoustic feature sets for improved performance, and the models estimate two time–frequency masks—the ideal ratio mask (IRM) and the ideal binary mask (IBM). Comprehensive evaluation using perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI) showed that the proposed LSTM architecture demonstrates enhanced speech intelligibility and perceptual quality. Composite measures further substantiated performance, considering residual noise distortion (Cbak) and speech distortion (Csig). The proposed model showed a 16.21% improvement in STOI and a 0.69 improvement in PESQ on the TIMIT database. Similarly, with the LibriSpeech database, the STOI and PESQ showed improvements of 16.41% and 0.71 over noisy mixtures. The proposed LSTM architecture outperforms deep neural networks (DNNs) in different stationary and nonstationary background noisy conditions. To train an automatic speech recognition (ASR) system on enhanced speech, the Kaldi toolkit is used for evaluating word error rate (WER). The proposed LSTM at the front-end notably reduced WERs, achieving a notable 15.13% WER across different noisy backgrounds.

Reducing the Computational Complexity of Two-Dimensional LSTMs

Convolutional Grid Long Short-Term Memory Recurrent Neural Network For Automatic Speech Recognition

Computational and Storage Efficient Quadratic Neurons for Deep Neural Networks

Bidirectional Grid Long Short-Term Memory (bigridlstm): A Method to Address Context-Sensitivity and Vanishing Gradient

Towards Efficient Recurrent Architectures: A Deep LSTM Neural Network Applied to Speech Enhancement and Recognition

Improved TDNNs Using Deep Kernels and Frequency Dependent Grid-RNNs.

Approximate Computing for Long Short Term Memory (LSTM) Neural Networks

Compact Feedforward Sequential Memory Networks for Large Vocabulary Continuous Speech Recognition

Advanced Recurrent Network-Based Hybrid Acoustic Models for Low Resource Speech Recognition

Improving speech recognition by revising gated recurrent units

High-Accuracy and Low-Latency Speech Recognition with Two-Head Contextual Layer Trajectory LSTM Model

On the Application and Compression of Deep Time Delay Neural Network for Embedded Statistical Parametric Speech Synthesis

Exploiting Symmetric Temporally Sparse BPTT for Efficient RNN Training

Deep causal speech enhancement and recognition using efficient long-short term memory Recurrent Neural Network

Modeling Speaker Variability Using Long Short-Term Memory Networks For Speech Recognition

Bidirectional LSTM with Extended Input Context

On the Effectiveness of Low-Rank Matrix Factorization for LSTM Model Compression

GhostRNN: Reducing State Redundancy in RNN with Cheap Operations

Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning

An Improved Time Feedforward Connections Recurrent Neural Networks

Multichannel Speech Enhancement Based on Time-Frequency Masking Using Subband Long Short-Term Memory