Abstract:Whisper, as a form of speech, is not sufficiently addressed by mainstream speech applications. This is due to the fact that systems built for normal speech do not work as expected for whispered speech. A first step to building a speech application that is inclusive of whispered speech, is the successful classification of whispered speech and normal speech. Such a front-end classification system is expected to have high accuracy and low computational overhead, which is the scope of this paper. One of the characteristics of whispered speech is the absence of the fundamental frequency (or pitch), and hence the pitch harmonics as well. The presence of the pitch and pitch harmonics in normal speech, and its absence in whispered speech, is evident in the spectral envelope of the Fourier transform. We observe that this characteristic is predominant in the first quarter of the spectrum, and exploit the same as a feature. We propose the use of one dimensional convolutional neural networks (1D-CNN) to capture these features from the quartered spectral envelope (QSE). The system yields an accuracy of 99.31% when trained and tested on the wTIMIT dataset, and 100% on the CHAINS dataset. The proposed feature is compared with Mel frequency cepstral coefficients (MFCC), a staple in the speech domain. The proposed classification system is also compared with the state-of-the-art system based on log-filterbank energy (LFBE) features trained on long short-term memory (LSTM) network. The proposed system based on 1D-CNN performs better than, or as good as, the state-of-the-art across multiple experiments. It also converges sooner, with lesser computational overhead. Finally, the proposed system is evaluated under the presence of white noise at various signal-to-noise ratios and found to be robust.

Reconstruction of Normal Speech from Whispered Speech Based on BP Neural Network

Attention-Guided Generative Adversarial Network for Whisper to Normal Speech Conversion

Whisper-to-speech Conversion Using Restricted Boltzmann Machine Arrays

End-to-End Whisper to Natural Speech Conversion using Modified Transformer Network

Speech Reconstruction from Silent Tongue and Lip Articulation By Pseudo Target Generation and Domain Adversarial Training

A novel BNMF-DNN based speech reconstruction method for speech quality evaluation under complex environments

Research on Speech Recognition Methods Based on Spectral Subtraction and Improved BP Neural Network Algorithm

Speech Reconstruction from Silent Lip and Tongue Articulation by Diffusion Models and Text-Guided Pseudo Target Generation

Deep Neural Network Based Noised Asian Speech Enhancement and Its Implementation on a Hearing Aid App.

Towards reconstructing intelligible speech from the human auditory cortex

Quartered Spectral Envelope and 1D-CNN-based Classification of Normally Phonated and Whispered Speech

Sequence-to-Sequence Voice Reconstruction for Silent Speech in a Tonal Language

Reconstructing Speech from Real-Time Articulatory MRI Using Neural Vocoders

WESPER: Zero-shot and Realtime Whisper to Normal Voice Conversion for Whisper-based Speech Interactions

A noise-robust voice conversion method with controllable background sounds

Exploiting Audio-Visual Features with Pretrained AV-HuBERT for Multi-Modal Dysarthric Speech Reconstruction

Two-stage and Self-supervised Voice Conversion for Zero-Shot Dysarthric Speech Reconstruction

Disentangling Content and Fine-Grained Prosody Information Via Hybrid ASR Bottleneck Features for Voice Conversion

An Optimized Neural Network Based Prosody Model of Chinese Speech Synthesis System

Neural2Speech: A Transfer Learning Framework for Neural-Driven Speech Reconstruction

Whispered-to-voiced Alaryngeal Speech Conversion with Generative Adversarial Networks