Abstract:In real-time applications, the aim of speech enhancement (SE) is to achieve optimal performance while ensuring computational efficiency and near-instant outputs. Many deep neural models have achieved optimal performance in terms of speech quality and intelligibility. However, formulating efficient and compact deep neural models for real-time processing on resource-limited devices remains a challenge. This study presents a compact neural model designed in a complex frequency domain for speech enhancement, optimized for resource-limited devices. The proposed model combines convolutional encoder–decoder and recurrent architectures to effectively learn complex mappings from noisy speech for real-time speech enhancement, enabling low-latency causal processing. Recurrent architectures such as Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), and Simple Recurrent Unit (SRU), are incorporated as bottlenecks to capture temporal dependencies and improve the performance of SE. By representing the speech in the complex frequency domain, the proposed model processes both magnitude and phase information. Further, this study extends the proposed models and incorporates attention-gate-based skip connections, enabling the models to focus on relevant information and dynamically weigh the important features. The results show that the proposed models outperform the recent benchmark models and obtain better speech quality and intelligibility. The proposed models show less computational load and deliver better results. This study uses the WSJ0 database where clean sentences from WSJ0 are mixed with different background noises to create noisy mixtures. The results show that STOI and PESQ are improved by 21.1% and 1.25 (41.5%) on the WSJ0 database whereas, on the VoiceBank+DEMAND database, STOI and PESQ are improved by 4.1% and 1.24 (38.6%) respectively. The extension of the models shows further improvement in STOI and PESQ in seen and unseen noisy conditions.

Neural Speech Synthesis on a Shoestring: Improving the Efficiency of LPCNet

Improving Opus Low Bit Rate Quality with Neural Speech Synthesis

Controllable Sequence-To-Sequence Neural TTS with LPCNET Backend for Real-time Speech Synthesis on CPU

Bunched LPCNet2: Efficient Neural Vocoders Covering Devices from Cloud to Edge

High quality, lightweight and adaptable TTS using LPCNet

LPCSE: Neural Speech Enhancement through Linear Predictive Coding

Improving LPCNet-based Text-to-Speech with Linear Prediction-structured Mixture Density Network

EfficientSpeech: An On-Device Text to Speech Model

Compact Deep Neural Networks for Real-Time Speech Enhancement on Resource-Limited Devices

Efficient Sparse Coding with the Adaptive Locally Competitive Algorithm for Speech Classification

A Hybrid Structure Speech Coding Scheme Based on MELPe and LPCNet

Ultra-lightweight Neural Differential DSP Vocoder For High Quality Speech Synthesis

SpeedySpeech: Efficient Neural Speech Synthesis

Towards Ultra-Low-Power Neuromorphic Speech Enhancement with Spiking-FullSubNet

TacoLPCNet: Fast and Stable TTS by Conditioning LPCNet on Mel Spectrogram Predictions

Natural speech re-synthesis from direct cortical recordings using a pre-trained encoder-decoder framework

Puffin: pitch-synchronous neural waveform generation for fullband speech on modest devices

A Perceptually-Motivated Approach for Low-Complexity, Real-Time Enhancement of Fullband Speech

Voice Synthesis Improvement by Machine Learning of Natural Prosody

The Design of an LPC Speech Synthesizer

Efficient High-Performance Bark-Scale Neural Network for Residual Echo and Noise Suppression