Abstract:The task of estimating the fundamental frequency of an audio signal, also known as pitch tracking, has been a long-standing research area in speech signal processing. Recently, there has been a growing interest in data-driven pitch tracking methods that leverage deep learning technologies. However, deep learning-based pitch tracking models often neglect the methodologies employed in well-established signal processing-based pitch tracking algorithms, which incorporate valuable prior knowledge. Motivated by this, we propose Neural RAPT, an interpretable neural network-based pitch tracking model that incorporates signal processing knowledge from RAPT. The proposed model consists of a front-end module and a back-end module. The front-end module adopts the U-Net structure, which inherently involves downsampling and upsampling processes similar to RAPT. To enhance the U-Net, we introduce a neural autocorrelation function (ACF) module using masked CNN, along with a normalization layer that models the sample pair-wise product in the normalized cross-correlation function (NCCF). The back-end module is based on the Transformer architecture. The model is evaluated by pitch tracking experiments on the PTDB-TUG database and noisy mixtures with the NOISEX-92 database at different SNRs. The experimental results demonstrate that incorporating algorithmic knowledge into the model design leads to improved performance. The proposed model inherits the advantages of high accuracy from traditional pitch tracking algorithms, while also benefiting from the noise robustness offered by neural network-based methods. Consequently, our model exhibits superiority over existing deep learning-based pitch tracking methods.

A Novel Pitch Extraction Based on Jointly Trained Deep BLSTM Recurrent Neural Networks with Bottleneck Features

RNN-BLSTM Based Multi-Pitch Estimation

An initial research: Towards accurate pitch extraction for speech synthesis based on BLSTM

Robust Multipitch Estimation Of Piano Sounds Using Deep Spiking Neural Networks

A pairwise algorithm for pitch estimation and speech separation using deep stacking network

Convolutional neural network for robust pitch determination.

Noise-Robust DSP-Assisted Neural Pitch Estimation with Very Low Complexity

Encoder-decoder based pitch tracking and joint model training for mandarin tone classification

A Novel Unified Framework for Speech Enhancement and Bandwidth Extension Based on Jointly Trained Neural Networks

A Pairwise Algorithm Using the Deep Stacking Network for Speech Separation and Pitch Estimation

Neural RAPT: deep learning-based pitch tracking with prior algorithmic knowledge instillation

Dual Attention Network for Pitch Estimation of Monophonic Music

A Robust and Low Computational Cost Pitch Estimation Method

Speech Bottleneck Feature Extraction Method Based on Overlapping Group Lasso Sparse Deep Neural Network

Dblstm-Based Multi-Task Learning for Pitch Transformation in Voice Conversion

A Study on the Robustness of Pitch Range Estimation from Brief Speech Segments

Multi-task Learning of Structured Output Layer Bidirectional LSTMS for Speech Synthesis

A Pitch-aware Speaker Extraction Serial Network

Singing voice separation and pitch extraction from monaural polyphonic audio music via DNN and adaptive pitch tracking

Pitch-Aware RNN-T for Mandarin Chinese Mispronunciation Detection and Diagnosis

Efficient pitch‐estimation network for edge devices