Abstract:Long short-term memory (LSTM) has proven effective in modeling sequential data. However, it may encounter challenges in accurately capturing long-term temporal dependencies. LSTM plays a central role in speech enhancement by effectively modeling and capturing temporal dependencies in speech signals. This paper introduces a variable-neurons-based LSTM designed for capturing long-term temporal dependencies by reducing neuron representation in layers with no loss of data. A skip connection between nonadjacent layers is added to prevent gradient vanishing. An attention mechanism in these connections highlights important features and spectral components. Our LSTM is inherently causal, making it well-suited for real-time processing without relying on future information. Training involves utilizing combined acoustic feature sets for improved performance, and the models estimate two time–frequency masks—the ideal ratio mask (IRM) and the ideal binary mask (IBM). Comprehensive evaluation using perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI) showed that the proposed LSTM architecture demonstrates enhanced speech intelligibility and perceptual quality. Composite measures further substantiated performance, considering residual noise distortion (Cbak) and speech distortion (Csig). The proposed model showed a 16.21% improvement in STOI and a 0.69 improvement in PESQ on the TIMIT database. Similarly, with the LibriSpeech database, the STOI and PESQ showed improvements of 16.41% and 0.71 over noisy mixtures. The proposed LSTM architecture outperforms deep neural networks (DNNs) in different stationary and nonstationary background noisy conditions. To train an automatic speech recognition (ASR) system on enhanced speech, the Kaldi toolkit is used for evaluating word error rate (WER). The proposed LSTM at the front-end notably reduced WERs, achieving a notable 15.13% WER across different noisy backgrounds.

Temporal Modeling Using Dilated Convolution and Gating for Voice-Activity-Detection

A Novel and Efficient Voice Activity Detector Using Shape Features of Speech Wave.

Voice Activity Detection Based on Time-Delay Neural Networks

Real-Time Causal Spectro-Temporal Voice Activity Detection Based on Convolutional Encoding and Residual Decoding

Formant Tracking Using Dilated Convolutional Networks Through Dense Connection with Gating Mechanism

Bimodal Recurrent Neural Network for Audiovisual Voice Activity Detection

Incorporating VAD into ASR System by Multi-task Learning

Multi-task Joint-Learning for Robust Voice Activity Detection

Waveform-based Voice Activity Detection Exploiting Fully Convolutional networks with Multi-Branched Encoders

Personal VAD: Speaker-Conditioned Voice Activity Detection

Voice activity detection using a local-global attention model

Boosted deep neural networks and multi-resolution cochleagram features for voice activity detection.

End-to-End Speaker-Dependent Voice Activity Detection

A Universal VAD Based on Jointly Trained Deep Neural Networks.

Boosting Contextual Information for Deep Neural Network Based Voice Activity Detection

Towards Efficient Recurrent Architectures: A Deep LSTM Neural Network Applied to Speech Enhancement and Recognition

A Real-Time Voice Activity Detection Based On Lightweight Neural

Voice activity detection using convolutive non-negative sparse coding

DNN-based Voice Activity Detection for Speaker Recognition

Efficient voice activity detection algorithm based on sub-band temporal envelope and sub-band long-term signal variability

CLIP-VAD: Exploiting Vision-Language Models for Voice Activity Detection