Abstract:Supervised speech enhancement has gained significantly from recent advancements in neural networks, especially due to their ability to non-linearly fit the diverse representations of target speech, such as waveform or spectrum. However, these direct-fitting solutions continue to face challenges with degraded speech and residual noise in hearing evaluations. By bridging the speech enhancement and the Information Bottleneck principle in this letter, we rethink a universal plug-and-play strategy and propose a Refining Underlying Information framework called RUI to rise to the challenges both in theory and practice. Specifically, we first transform the objective of speech enhancement into an incremental convergence problem of mutual information between comprehensive speech characteristics and individual speech characteristics, e.g., spectral and acoustic characteristics. By doing so, compared with the existing direct-fitting solutions, the underlying information stems from the conditional entropy of acoustic characteristic given spectral characteristics. Therefore, we design a dual-path multiple refinement iterator based on the chain rule of entropy to refine this underlying information for further approximating target speech. Experimental results on DNS-Challenge dataset show that our solution consistently improves 0.3+ PESQ score over baselines, with only additional 1.18 M parameters. The source code is available at <a class="link-external link-https" href="https://github.com/caoruitju/RUI_SE" rel="external noopener nofollow">this https URL</a>.

Supervised Monaural Speech Enhancement Using Complementary Joint Sparse Representations.

Adaptive Speech Enhancement Using Sparse Prior Information.

Feature Denoising Using Joint Sparse Representation for In-Car Speech Recognition.

Speech Enhancement with a GSC-like Structure Employing Sparse Coding

A Semi-Supervised Complementary Joint Training Approach for Low-Resource Speech Recognition

Joint Training for Simultaneous Speech Denoising and Dereverberation with Deep Embedding Representations

A Lightweight and Real-Time Binaural Speech Enhancement Model with Spatial Cues Preservation

A Refining Underlying Information Framework for Monaural Speech Enhancement

Study on Sparse Coding Speech Enhancement

Joint sparse representation based cepstral-domain dereverberation for distant-talking speech recognition

Improving Monaural Speech Enhancement by Mapping to Fixed Simulation Space With Knowledge Distillation

A Two-Stage Training Framework for Joint Speech Compression and Enhancement

Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation

Mixture Encoder for Joint Speech Separation and Recognition

Injecting Spatial Information for Monaural Speech Enhancement via Knowledge Distillation

Manifold constrained joint sparse learning via non-convex regularization

Multichannel blind speech source separation with a disjoint constraint source model

Jointly Recognizing Speech and Singing Voices Based on Multi-Task Audio Source Separation

Facial Expression Recognition Via Weighted Group Sparsity.

On the Efficacy and Noise-Robustness of Jointly Learned Speech Emotion and Automatic Speech Recognition

Multimodal image fusion with adaptive joint sparsity model