Abstract:The presence of background noise or competing talkers is one of the main communication challenges for cochlear implant (CI) users in speech understanding in naturalistic spaces. These external factors distort the time-frequency (T-F) content including magnitude spectrum and phase of speech signals. While most existing speech enhancement (SE) solutions focus solely on enhancing the magnitude response, recent research highlights the importance of phase in perceptual speech quality. Motivated by multi-task machine learning, this study proposes a deep complex convolution transformer network (DCCTN) for complex spectral mapping, which simultaneously enhances the magnitude and phase responses of speech. The proposed network leverages a complex-valued U-Net structure with a transformer within the bottleneck layer to capture sufficient low-level detail of contextual information in the T-F domain. To capture the harmonic correlation in speech, DCCTN incorporates a frequency transformation block in the encoder structure of the U-Net architecture. The DCCTN learns a complex transformation matrix to accurately recover speech in the T-F domain from a noisy input spectrogram. Experimental results demonstrate that the proposed DCCTN outperforms existing model solutions such as the convolutional recurrent network (CRN), deep complex convolutional recurrent network (DCCRN), and gated convolutional recurrent network (GCRN) in terms of objective speech intelligibility and quality, both for seen and unseen noise conditions. To evaluate the effectiveness of the proposed SE solution, a formal listener evaluation involving four CI recipients was conducted. Results indicate a significant improvement in speech intelligibility performance for CI recipients in noisy environments. Additionally, DCCTN demonstrates the capability to suppress highly non-stationary noise without introducing musical artifacts commonly observed in conventional SE methods.

Single Channel Speech Enhancement Based on Temporal Convolutional Network

TFCN: Temporal-Frequential Convolutional Network for Single-Channel Speech Enhancement

Audio-Visual Speech Enhancement with Deep Multi-modality Fusion

Multi-stage Progressive Learning-Based Speech Enhancement Using Time–Frequency Attentive Squeezed Temporal Convolutional Networks

Speech Enhancement Algorithm Based on a Convolutional Neural Network Reconstruction of the Temporal Envelope of Speech in Noisy Environments

Improving Speech Enhancement by Integrating Inter-Channel and Band Features with Dual-branch Conformer

Speech Enhancement Using Multi-Stage Self-Attentive Temporal Convolutional Networks

Time domain speech enhancement with CNN and time-attention transformer

Utterance Weighted Multi-Dilation Temporal Convolutional Networks for Monaural Speech Dereverberation

Multi-Loss Convolutional Network with Time-Frequency Attention for Speech Enhancement

PCF: ECAPA-TDNN with Progressive Channel Fusion for Speaker Verification

Target Speaker Extraction Using Attention-Enhanced Temporal Convolutional Network

Inter-channel Conv-TasNet for multichannel speech enhancement

Decoupling-style monaural speech enhancement with a triple-branch cross-domain fusion network

Multichannel Speech Enhancement without Beamforming

PCNN: A Lightweight Parallel Conformer Neural Network for Efficient Monaural Speech Enhancement

TMS: Temporal multi-scale in time-delay neural network for speaker verification

Efficient Encoder-Decoder and Dual-Path Conformer for Comprehensive Feature Learning in Speech Enhancement

Temporal-Channel Modeling in Multi-head Self-Attention for Synthetic Speech Detection

CompNet: Complementary Network for Single-Channel Speech Enhancement.

Speech Enhancement for Cochlear Implant Recipients using Deep Complex Convolution Transformer with Frequency Transformation