Abstract:Whispered speech is a special voicing style of speech that is employed publicly to protect speech information. It is also the primary pronunciation form for aphonic individuals with laryngectomy for oral communication. Converting whispered speech to normal-voiced speech can significantly improve speech quality and/or speech intelligibility for whisper perception or recognition. Due to the significant voicing style difference between normal speech and whispered speech, it is still a major challenge to estimate normal-voiced speech from its whispered counterpart. Existing whisper-to-normal speech conversion methods aim to learn a nonlinear function of features between whispered speech and its normal counterpart, and the converted normal speech is reconstructed with features selected by the learned function from the training data space. These methods may produce a discontinuous spectrum in successive frames, thus decreasing the speech quality and/or intelligibility of the converted normal speech. This paper proposes a novel generative model (AGAN-W2SC) for whisper-to-normal speech conversion. Unlike the feature mapping model, the proposed AGAN-W2SC model generates a normal speech spectrum from a whispered spectrum. To make the generated spectrum more similar to the reference normal speech, the inner-feature coherence of a whisper as well as the inter-feature coherence between whispered speech and its normal counterpart is modeled in the proposed AGAN-W2SC model. Specifically, a self-attention mechanism is introduced to capture the inner-spectrum structure while a Siamese neural network is adopted to capture the interspectrum structure in the cross-domain. Additionally, the proposed model adopts identity mapping to preserve linguistic information. The proposed AGAN-W2SC is parallel data-free and can be trained at the frame level. Experimental results on whisper-to-normal speech conversion demonstrate the superior performance and effectiveness of the proposed AGAN-W2SC method over all the compared competing methods in terms of speech quality and intelligibility

Learning Stage-wise GANs for Whistle Extraction in Time-Frequency Spectrograms

WhistleGAN for Biomimetic Underwater Acoustic Covert Communication

Using deep learning to track time × frequency whistle contours of toothed whales without human-annotated training data

Low-Resource Generation Method for Few-Shot Dolphin Whistle Signal Based on Generative Adversarial Network

Automated Classification of Tursiops Aduncus Whistles Based on a Depth-Wise Separable Convolutional Neural Network and Data Augmentation.

Robust unsupervised Tursiops aduncus whistle-event detection using gammatone multi-channel Savitzky-Golay based whistle enhancement

Robust unsupervised <i>Tursiops aduncus</i> whistle enhancement based on complete ensembled empirical optimal envelope local mean decomposition with adaptive noise

Study of GANs for Noisy Speech Simulation from Clean Speech

Dolphin Whistle Signal Enhancement Using Nonconvex Time-Frequency Regularized Overlapping Group Shrinkage in Warm Shallow Water

Snore-GANs: Improving Automatic Snore Sound Classification With Synthesized Data

Automated Classification of Dolphin Whistles Based on the Convolutional Neural Network

Enhancing Seismic Data Denoising Through Multiscale Analysis Across Transformer and GAN

GANs for Children: A Generative Data Augmentation Strategy for Children Speech Recognition

Feature-Matching Speech Denoising GANs via Progressive Training.

Spectral Entropy Based Dolphin Whistle Detection Algorithm and Its Possible Application for Biologically Inspired Communication

Automatic Segmentation and Identification of Whistles Produced by Dolphins.

Bidirectional Generative Adversarial Representation Learning for Natural Stimulus Synthesis

A Novel Tracker of Adaptive Directional Ridge Separation and Prediction for Detecting Whistles

A Novel Attention-Guided Generative Adversarial Network for Whisper-to-Normal Speech Conversion

Generative Adversarial Networks Based Data Augmentation for Noise Robust Speech Recognition

Noise Modeling to Build Training Sets for Robust Speech Enhancement