The Effect of Training Dataset Size on Discriminative and Diffusion-Based Speech Enhancement Systems

Philippe Gonzalez,Zheng-Hua Tan,Jan Østergaard,Jesper Jensen,Tommy Sonne Alstrøm,Tobias May

2024-06-10

Abstract:The performance of deep neural network-based speech enhancement systems typically increases with the training dataset size. However, studies that investigated the effect of training dataset size on speech enhancement performance did not consider recent approaches, such as diffusion-based generative models. Diffusion models are typically trained with massive datasets for image generation tasks, but whether this is also required for speech enhancement is unknown. Moreover, studies that investigated the effect of training dataset size did not control for the data diversity. It is thus unclear whether the performance improvement was due to the increased dataset size or diversity. Therefore, we systematically investigate the effect of training dataset size on the performance of popular state-of-the-art discriminative and diffusion-based speech enhancement systems. We control for the data diversity by using a fixed set of speech utterances, noise segments and binaural room impulse responses to generate datasets of different sizes. We find that the diffusion-based systems do not benefit from increasing the training dataset size as much as the discriminative systems. They perform the best relative to the discriminative systems with datasets of 10 h or less, but they are outperformed by the discriminative systems with datasets of 100 h or more.

Audio and Speech Processing

What problem does this paper attempt to address?

The paper attempts to address the impact of training dataset size on the performance of discriminative and diffusion-based speech enhancement systems. Specifically, the paper focuses on the following points:

1. **Applicability of diffusion models in speech enhancement**: Although diffusion models typically require large-scale datasets for training in image generation tasks, their performance in speech enhancement is still unclear. The paper aims to explore whether diffusion models also need a similar amount of data to achieve optimal performance.

2. **Independent impact of dataset size and diversity**: Previous studies investigating the impact of training dataset size on speech enhancement performance often did not control for data diversity. Therefore, it is unclear whether performance improvements are due to an increase in dataset size or data diversity. The paper controls data diversity by using a fixed set of speech, noise, and binaural room impulse responses (BRIR) to generate datasets of different sizes, thereby independently evaluating the impact of dataset size.

3. **Comparison of modern deep learning models**: The paper selects three popular discriminative speech enhancement systems (Conv-TasNet, DCCRN, MANNER) and three diffusion-based speech enhancement systems (SGMSE+, SGMSE+M, SGMSE+Mcos Heun) to systematically compare their performance on datasets of different sizes.

In summary, the main objective of the paper is to systematically study the impact of training dataset size on the performance of modern discriminative and diffusion-based speech enhancement systems and to explore the performance differences of these systems under different dataset sizes.