Abstract:The presence of background noise or competing talkers is one of the main communication challenges for cochlear implant (CI) users in speech understanding in naturalistic spaces. These external factors distort the time-frequency (T-F) content including magnitude spectrum and phase of speech signals. While most existing speech enhancement (SE) solutions focus solely on enhancing the magnitude response, recent research highlights the importance of phase in perceptual speech quality. Motivated by multi-task machine learning, this study proposes a deep complex convolution transformer network (DCCTN) for complex spectral mapping, which simultaneously enhances the magnitude and phase responses of speech. The proposed network leverages a complex-valued U-Net structure with a transformer within the bottleneck layer to capture sufficient low-level detail of contextual information in the T-F domain. To capture the harmonic correlation in speech, DCCTN incorporates a frequency transformation block in the encoder structure of the U-Net architecture. The DCCTN learns a complex transformation matrix to accurately recover speech in the T-F domain from a noisy input spectrogram. Experimental results demonstrate that the proposed DCCTN outperforms existing model solutions such as the convolutional recurrent network (CRN), deep complex convolutional recurrent network (DCCRN), and gated convolutional recurrent network (GCRN) in terms of objective speech intelligibility and quality, both for seen and unseen noise conditions. To evaluate the effectiveness of the proposed SE solution, a formal listener evaluation involving four CI recipients was conducted. Results indicate a significant improvement in speech intelligibility performance for CI recipients in noisy environments. Additionally, DCCTN demonstrates the capability to suppress highly non-stationary noise without introducing musical artifacts commonly observed in conventional SE methods.

VHF Speech Enhancement Based on Transformer

SETransformer: Speech Enhancement Transformer

Audio-Visual Speech Enhancement with Deep Multi-modality Fusion

Dual-path transformer-based network with equalization-generation components prediction for flexible vibrational sensor speech enhancement in the time domain

VSET: A MULTIMODAL TRANSFORMER FOR VISUAL SPEECH ENHANCEMENT

Efficient Monaural Speech Enhancement using Spectrum Attention Fusion

THLNet: two-stage heterogeneous lightweight network for monaural speech enhancement

Time domain speech enhancement with CNN and time-attention transformer

VHF Speech Recognition Model Based on Improved Conformer Structure

A Hybrid Deep-Learning Approach for Single Channel HF-SSB Speech Enhancement

Lightweight Dynamic Sparse Transformer for Monaural Speech Enhancement

Speech Enhancement for Cochlear Implant Recipients using Deep Complex Convolution Transformer with Frequency Transformation

Improving Deep Neural Network Based Speech Enhancement in Low SNR Environments

Research on Speech Enhancement based on Full-scale Connection

Lightweight Multi-Axial Transformer with Frequency Prompt for Single Channel Speech Enhancement.

ME-TransNet: A Modal-Enhanced Transformer Denoising Network for Attenuating Low-Frequency Swell Noise

Channel Estimation Method Based on Transformer in High Dynamic Environment

Single Channel Speech Enhancement Using Temporal Convolutional Recurrent Neural Networks.

DCHT: Deep Complex Hybrid Transformer for Speech Enhancement

Monaural Speech Enhancement Based on Spectrogram Decomposition for Convolutional Neural Network-sensitive Feature Extraction

CST-UNet: Cross Swin Transformer Enhanced U-Net with Masked Bottleneck for Single-Channel Speech Enhancement