Abstract:Extracting pitch information from music recordings is a challenging but important problem in music signal processing. Frame-wise transcription or multi-pitch estimation aims for detecting the simultaneous activity of pitches in polyphonic music recordings and has recently seen major improvements thanks to deep-learning techniques, with a variety of proposed network architectures. In this paper, we realize different architectures based on CNNs, the U-net structure, and self-attention components. We propose several modifications to these architectures including self-attention modules for skip connections, recurrent layers to replace the self-attention, and a multi-task strategy with simultaneous prediction of the degree of polyphony. We compare variants of these architectures in different sizes for multi-pitch estimation, focusing on Western classical music beyond the piano-solo scenario using the MusicNet and Schubert Winterreise datasets. Our experiments indicate that most architectures yield competitive results and that larger model variants seem to be beneficial. However, we find that these results substantially depend on randomization effects and the particular choice of the training-test split, which questions the claim of superiority for particular architectures given only small improvements. We therefore investigate the influence of dataset splits in the presence of several movements of a work cycle (cross-version evaluation) and propose a best-practice splitting strategy for MusicNet, which weakens the influence of individual test tracks and suppresses overfitting to specific works and recording conditions. A final evaluation on a mixed dataset suggests that improvements on one specific dataset do not necessarily generalize to other scenarios, thus emphasizing the need for further high-quality multi-pitch datasets in order to reliably measure progress in music transcription tasks.

RNN-BLSTM Based Multi-Pitch Estimation

A Novel Pitch Extraction Based on Jointly Trained Deep BLSTM Recurrent Neural Networks with Bottleneck Features

Robust Multipitch Estimation Of Piano Sounds Using Deep Spiking Neural Networks

An initial research: Towards accurate pitch extraction for speech synthesis based on BLSTM

MF-PAM: Accurate Pitch Estimation through Periodicity Analysis and Multi-level Feature Fusion

Dual Attention Network for Pitch Estimation of Monophonic Music

A Robust and Low Computational Cost Pitch Estimation Method

Deep-Learning Architectures for Multi-Pitch Estimation: Towards Reliable Evaluation

A pairwise algorithm for pitch estimation and speech separation using deep stacking network

Speech Enhancement Based On Analysis Synthesis Framework With Improved Pitch Estimation And Spectral Envelope Enhancement

A Music Cognition–Guided Framework for Multi-pitch Estimation

Multi-task Learning of Structured Output Layer Bidirectional LSTMS for Speech Synthesis

Multi-task Melody Extraction Using Feature Optimization and CRNN-CRF.

Computationality Efficient Multi-Pitch Estimation Using Sparsity

Single-channel speech separation integrating pitch information based on a multi task learning framework

Toward Fully Self-Supervised Multi-Pitch Estimation

Encoder-decoder based pitch tracking and joint model training for mandarin tone classification

Neural RAPT: deep learning-based pitch tracking with prior algorithmic knowledge instillation

A Study on the Robustness of Pitch Range Estimation from Brief Speech Segments

Pitch estimation using mean shift algorithm on multitaper spectrum of noisy speech

Pitch-Aware RNN-T for Mandarin Chinese Mispronunciation Detection and Diagnosis