Abstract:Long short-term memory (LSTM) has proven effective in modeling sequential data. However, it may encounter challenges in accurately capturing long-term temporal dependencies. LSTM plays a central role in speech enhancement by effectively modeling and capturing temporal dependencies in speech signals. This paper introduces a variable-neurons-based LSTM designed for capturing long-term temporal dependencies by reducing neuron representation in layers with no loss of data. A skip connection between nonadjacent layers is added to prevent gradient vanishing. An attention mechanism in these connections highlights important features and spectral components. Our LSTM is inherently causal, making it well-suited for real-time processing without relying on future information. Training involves utilizing combined acoustic feature sets for improved performance, and the models estimate two time–frequency masks—the ideal ratio mask (IRM) and the ideal binary mask (IBM). Comprehensive evaluation using perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI) showed that the proposed LSTM architecture demonstrates enhanced speech intelligibility and perceptual quality. Composite measures further substantiated performance, considering residual noise distortion (Cbak) and speech distortion (Csig). The proposed model showed a 16.21% improvement in STOI and a 0.69 improvement in PESQ on the TIMIT database. Similarly, with the LibriSpeech database, the STOI and PESQ showed improvements of 16.41% and 0.71 over noisy mixtures. The proposed LSTM architecture outperforms deep neural networks (DNNs) in different stationary and nonstationary background noisy conditions. To train an automatic speech recognition (ASR) system on enhanced speech, the Kaldi toolkit is used for evaluating word error rate (WER). The proposed LSTM at the front-end notably reduced WERs, achieving a notable 15.13% WER across different noisy backgrounds.

FPGA Implementation of LSTM Based on Automatic Speech Recognition

A Near Memory Computing FPGA Architecture for Neural Network Acceleration

A Spiking LSTM Accelerator for Automatic Speech Recognition Application Based on FPGA

FPGA-based Accelerator for Long Short-Term Memory Recurrent Neural Networks

A Power-Efficient Accelerator Based on FPGAs for LSTM Network

C-LSTM: Enabling Efficient LSTM Using Structured Compression Techniques on FPGAs

Ese: Efficient Speech Recognition Engine with Sparse Lstm on Fpga

Implementation and Optimization of the Accelerator Based on FPGA Hardware for LSTM Network

E-RNN: Design Optimization for Efficient Recurrent Neural Networks in FPGAs

Design and Implementation of Intelligent Speech Recognition System Based on FPGA

FPGA Acceleration of Recurrent Neural Network Based Language Model

A Low-Latency and Low-Complexity Hardware Architecture for CTC Beam Search Decoding

Towards Efficient Recurrent Architectures: A Deep LSTM Neural Network Applied to Speech Enhancement and Recognition

Efficient and Effective Sparse LSTM on FPGA with Bank-Balanced Sparsity

A low-latency LSTM accelerator using balanced sparsity based on FPGA

Acceleration of LSTM with Structured Pruning Method on FPGA

A Highly Configurable 7.62gop/s Hardware Implementation for LSTM

A Compact and Configurable Long Short-Term Memory Neural Network Hardware Architecture.

Design and Implementation of Embedded Real-Time English Speech Recognition System Based on Big Data Analysis

The implementation of a Deep Recurrent Neural Network Language Model on a Xilinx FPGA

FPGA Deployment of LFADS for Real-time Neuroscience Experiments