Abstract:Single-channel, speaker-independent speech separation methods have recently seen great progress. However, the accuracy, latency, and computational cost of such methods remain insufficient. The majority of the previous methods have formulated the separation problem through the time–frequency representation of the mixed signal, which has several drawbacks, including the decoupling of the phase and magnitude of the signal, the suboptimality of time–frequency representation for speech separation, and the long latency in calculating the spectrograms. To address these shortcomings, we propose a fully convolutional time-domain audio separation network Conv-TasNet, a deep learning framework for end-to-end time-domain speech separation. Conv-TasNet uses a linear encoder to generate a representation of the speech waveform optimized for separating individual speakers. Speaker separation is achieved by applying a set of weighting functions masks to the encoder output. The modified encoder representations are then inverted back to the waveforms using a linear decoder. The masks are found using a temporal convolutional network consisting of stacked one-dimensional dilated convolutional blocks, which allows the network to model the long-term dependencies of the speech signal while maintaining a small model size. The proposed Conv-TasNet system significantly outperforms previous time–frequency masking methods in separating two- and three-speaker mixtures. Additionally, Conv-TasNet surpasses several ideal time–frequency magnitude masks in two-speaker speech separation as evaluated by both objective distortion measures and subjective quality assessment by human listeners. Finally, Conv-TasNet has a significantly smaller model size and a shorter minimum latency, making it a suitable solution for both offline and real-time speech separation applications. This study, therefore, represents a major step toward the realization of speech separation systems for real-world speech processing technologies.

Toward Speech Separation in The Pre-Cocktail Party Problem with TasTas

Toward the Pre-Cocktail Party Problem with TasTas+.

Toward the pre-cocktail party problem with TasTas$+$

Speech Separation Based on Multi-Stage Elaborated Dual-Path Deep BiLSTM with Auxiliary Identity Loss.

LaFurca: Iterative Multi-Stage Refined End-to-End Monaural Speech Separation Based on Context-Aware Dual-Path Deep Parallel Inter-Intra Bi-LSTM

La Furca: Iterative Context-Aware End-to-End Monaural Speech Separation Based on Dual-Path Deep Parallel Inter-Intra Bi-LSTM with Attention.

Real-time binaural speech separation with preserved spatial cues

TaSNet: Time-Domain Audio Separation Network for Real-Time, Single-Channel Speech Separation

LaFurca: Iterative Refined Speech Separation Based on Context-Aware Dual-Path Parallel Bi-LSTM

Beam-Guided TasNet: an Iterative Speech Separation Framework with Multi-Channel Output

A comprehensive study of speech separation: spectrogram vs waveform separation

A Distinct Synthesizer Convolutional Tasnet For Singing Voice Separation

Speaker and Direction Inferred Dual-channel Speech Separation

Conv-TasNet: Surpassing Ideal Time–Frequency Magnitude Masking for Speech Separation

Localization Based Stereo Speech Separation Using Deep Networks.

On End-to-end Multi-channel Time Domain Speech Separation in Reverberant Environments

Reverberant Speech Separation with Probabilistic Time-Frequency Masking for B-format Recordings.

End-to-end Music-mixed Speech Recognition

End-to-End Monaural Speech Separation with Multi-Scale Dynamic Weighted Gated Dilated Convolutional Pyramid Network

Deep Attention Gated Dilated Temporal Convolutional Networks with Intra-Parallel Convolutional Modules for End-to-End Monaural Speech Separation

Separate And Diffuse: Using a Pretrained Diffusion Model for Improving Source Separation