Abstract:The presence of background noise or competing talkers is one of the main communication challenges for cochlear implant (CI) users in speech understanding in naturalistic spaces. These external factors distort the time-frequency (T-F) content including magnitude spectrum and phase of speech signals. While most existing speech enhancement (SE) solutions focus solely on enhancing the magnitude response, recent research highlights the importance of phase in perceptual speech quality. Motivated by multi-task machine learning, this study proposes a deep complex convolution transformer network (DCCTN) for complex spectral mapping, which simultaneously enhances the magnitude and phase responses of speech. The proposed network leverages a complex-valued U-Net structure with a transformer within the bottleneck layer to capture sufficient low-level detail of contextual information in the T-F domain. To capture the harmonic correlation in speech, DCCTN incorporates a frequency transformation block in the encoder structure of the U-Net architecture. The DCCTN learns a complex transformation matrix to accurately recover speech in the T-F domain from a noisy input spectrogram. Experimental results demonstrate that the proposed DCCTN outperforms existing model solutions such as the convolutional recurrent network (CRN), deep complex convolutional recurrent network (DCCRN), and gated convolutional recurrent network (GCRN) in terms of objective speech intelligibility and quality, both for seen and unseen noise conditions. To evaluate the effectiveness of the proposed SE solution, a formal listener evaluation involving four CI recipients was conducted. Results indicate a significant improvement in speech intelligibility performance for CI recipients in noisy environments. Additionally, DCCTN demonstrates the capability to suppress highly non-stationary noise without introducing musical artifacts commonly observed in conventional SE methods.

SETransformer: Speech Enhancement Transformer

VHF Speech Enhancement Based on Transformer

Lightweight Dynamic Sparse Transformer for Monaural Speech Enhancement

VSET: A MULTIMODAL TRANSFORMER FOR VISUAL SPEECH ENHANCEMENT

Speech Enhancement for Cochlear Implant Recipients using Deep Complex Convolution Transformer with Frequency Transformation

SpeechFormer: A Hierarchical Efficient Framework Incorporating the Characteristics of Speech

SpeechFormer++: A Hierarchical Efficient Framework for Paralinguistic Speech Processing

ZTrans: A new Transformer for Speech Enhancement

DCHT: Deep Complex Hybrid Transformer for Speech Enhancement

DBT-Net: Dual-branch Federative Magnitude and Phase Estimation with Attention-in-attention Transformer for Monaural Speech Enhancement

The Speechtransformer for Large-scale Mandarin Chinese Speech Recognition.

DPATD: Dual-Phase Audio Transformer for Denoising

Transformer with Bidirectional Decoder for Speech Recognition

Dual-Branch Attention-In-Attention Transformer for Single-Channel Speech Enhancement

Lightweight Multi-Axial Transformer with Frequency Prompt for Single Channel Speech Enhancement.

Synchronous Transformers for End-to-End Speech Recognition.

End-to-End Whisper to Natural Speech Conversion using Modified Transformer Network

TENET: A Time-reversal Enhancement Network for Noise-robust ASR

Efficient Monaural Speech Enhancement using Spectrum Attention Fusion

Speech-Transformer: A No-Recurrence Sequence-to-Sequence Model for Speech Recognition

Speech enhancement deep-learning architecture for efficient edge processing