Abstract:Single-channel, speaker-independent speech separation methods have recently seen great progress. However, the accuracy, latency, and computational cost of such methods remain insufficient. The majority of the previous methods have formulated the separation problem through the time–frequency representation of the mixed signal, which has several drawbacks, including the decoupling of the phase and magnitude of the signal, the suboptimality of time–frequency representation for speech separation, and the long latency in calculating the spectrograms. To address these shortcomings, we propose a fully convolutional time-domain audio separation network Conv-TasNet, a deep learning framework for end-to-end time-domain speech separation. Conv-TasNet uses a linear encoder to generate a representation of the speech waveform optimized for separating individual speakers. Speaker separation is achieved by applying a set of weighting functions masks to the encoder output. The modified encoder representations are then inverted back to the waveforms using a linear decoder. The masks are found using a temporal convolutional network consisting of stacked one-dimensional dilated convolutional blocks, which allows the network to model the long-term dependencies of the speech signal while maintaining a small model size. The proposed Conv-TasNet system significantly outperforms previous time–frequency masking methods in separating two- and three-speaker mixtures. Additionally, Conv-TasNet surpasses several ideal time–frequency magnitude masks in two-speaker speech separation as evaluated by both objective distortion measures and subjective quality assessment by human listeners. Finally, Conv-TasNet has a significantly smaller model size and a shorter minimum latency, making it a suitable solution for both offline and real-time speech separation applications. This study, therefore, represents a major step toward the realization of speech separation systems for real-world speech processing technologies.

Nonlinear Residual Echo Suppression Based on Multi-Stream Conv-TasNet.

Automatic Auditory Streaming Restores Missing Temporal Modulations in Echoic Speech

Residual acoustic echo suppression based on efficient multi-task convolutional neural network

Acoustic Echo Cancellation by Combining Adaptive Digital Filter and Recurrent Neural Network

An Efficient Post-Processing Filter for Multi-Channel Acoustic Echo Cancellation System

End-to-End Complex-Valued Multidilated Convolutional Neural Network for Joint Acoustic Echo Cancellation and Noise Suppression

Deep Residual Echo Suppression with A Tunable Tradeoff Between Signal Distortion and Echo Suppression

Nonlinear Acoustic Echo Cancellation with Deep Learning

Deep Echo Cancellation Algorithm Based on Time-Frequency Domain Combination

Conv-TasNet: Surpassing Ideal Time–Frequency Magnitude Masking for Speech Separation

Semi-blind source separation using convolutive transfer function for nonlinear acoustic echo cancellation

Beam-Guided TasNet: an Iterative Speech Separation Framework with Multi-Channel Output

A Robust and Cascaded Acoustic Echo Cancellation Based on Deep Learning

Adaptive Acoustic Echo Cancellation System Without Double-Talk Detection Using VSS-NLMS Algorithm

Inter-channel Conv-TasNet for multichannel speech enhancement

Double Branches and Stages Neural Network for Joint Acoustic Echo and Noise Suppression

Deep Multi-task Network for Delay Estimation and Echo Cancellation

A Robust Acoustic Echo Canceller for Noisy Environment

Double-Talk Detection-Aided Residual Echo Suppression via Spectrogram Masking and Refinement

Efficient High-Performance Bark-Scale Neural Network for Residual Echo and Noise Suppression

Improving Acoustic Echo Cancellation by Mixing Speech Local and Global Features with Transformer