Abstract:Singing melody extraction from polyphonic musical audio is one of the most challenging tasks in music information retrieval (MIR). Recently, data-driven methods based on convolutional neural networks (CNNs) have achieved great success for this task. In the literature, harmonic relationship has been proven crucial for this task. However, few existing CNN-based singing melody extraction methods consider the harmonic relationship in the training stage. The state-of-the-art CNN based methods are not capable of capturing such long-dependency harmonic relationship due to limited receptive field and unacceptable computation cost. In this paper, we introduce a neural harmonic-aware network with gated attentive fusion (NHAN-GAF) for singing melody extraction. Specifically, in the 2-D spectrograms modeling branch, we propose to employ multiple parallel 1-D CNN kernels to capture the harmonic relations between 1–2 octaves along the frequency axis in the spectrogram. Considering the advantage of jointly using Time–Frequency (T-F) domain and time domain information, we use two-branch neural nets to learn discriminative representation for this task. A novel gated attentive fusion (GAF) network is suggested to encode potential correlations between the two branches and fuse the descriptors learned from raw waveform and T-F spectrograms. Moreover, the idea of GAF can be exploited to the multimedia applications with multimodal analysis. With the two proposed components, our proposed model is capable of learning the harmonic relationship in the spectrogram and better capturing the contextual but discriminative features for singing melody extraction. We use part of the vocal tracks of the RWC dataset and MIR-1 K dataset to train the model and evaluate the performance of the proposed model on the ADC 2004, MIREX 05 and MedleyDB datasets. The experimental results show that the proposed method outperforms the state-of-the-art ones.

Signal Feature Extraction of Music Melody Based on Deep Learning

Music Waveform Analysis Based on SOM Neural Network and Big Data

Music Feature Extraction and Classification Algorithm Based on Deep Learning

Melody Extraction Based on Deep Harmonic Neural Network

Frequency-Temporal Attention Network for Singing Melody Extraction

Melody Extraction from Polyphonic Music by Deep Learning Approaches: A Review

A Multi-task Learning Approach for Melody Extraction

Harmonic Classification with Enhancing Music Using Deep Learning Techniques

Vocal Melody Extraction Using Attention U-net and Voice Detection Model

Music Similarity Detection Guided by Deep Learning Model

Efficient Melody Extraction Based On Extreme Learning Machine

Singing Melody Extraction Based on Combined Frequency-Temporal Attention and Attentional Feature Fusion with Self-Attention

An Evaluation of Deep Neural Network Models for Music Classification Using Spectrograms

Stereo Feature Enhancement and Temporal Information Extraction Network for Automatic Music Transcription

Multi-task Melody Extraction Using Feature Optimization and CRNN-CRF.

A neural harmonic-aware network with gated attentive fusion for singing melody extraction

Towards Improving Harmonic Sensitivity and Prediction Stability for Singing Melody Extraction

Music Track Recommendation Using Deep-CNN and Mel Spectrograms

Monophonic Singing Voice Separation Based on Deep Learning

MIDI MELODY EXTRACTION BASED ON IMPROVED NEURAL NETWORK

Construction of Music Intelligent Creation Model Based on Convolutional Neural Network