Abstract:Melody extraction is a vital music information retrieval task among music researchers for its potential applications in education pedagogy and the music industry. Melody extraction is a notoriously challenging task due to the presence of background instruments. Also, often melodic source exhibits similar characteristics to that of the other instruments. The interfering background accompaniment with the vocals makes extracting the melody from the mixture signal much more challenging. Until recently, classical signal processing-based melody extraction methods were quite popular among melody extraction researchers. The ability of the deep learning models to model large-scale data and the ability of the models to learn automatic features by exploiting spatial and temporal dependencies inspired many researchers to adopt deep learning models for melody extraction. In this paper, an attempt has been made to review the up-to-date data-driven deep learning approaches for melody extraction from polyphonic music. The available deep models have been categorized based on the type of neural network used and the output representation they use for predicting melody. Further, the architectures of the 25 melody extraction models are briefly presented. The loss functions used to optimize the model parameters of the melody extraction models are broadly categorized into four categories and briefly describe the loss functions used by various melody extraction models. Also, the various input representations adopted by the melody extraction models and the parameter settings are deeply described. A section describing the explainability of the block-box melody extraction deep neural networks is included. The performance of 25 melody extraction methods is compared. The possible future directions to explore/improve the melody extraction methods are also presented in the paper.

A multimodal deep learning algorithm for polyphonic music applied to music sentiment analysis and generation

Music Waveform Analysis Based on SOM Neural Network and Big Data

Real-Time Human-Music Emotional Interaction Based on Deep Learning and Multimodal Sentiment Analysis

MMD-MII Model: A Multilayered Analysis and Multimodal Integration Interaction Approach Revolutionizing Music Emotion Classification

Towards A Hybrid Deep-Learning Method For Music Classification And Similarity Measurement

Real-time Human-Music Emotional Interaction Based on Multimodal Analysis

A Music Cognition Based Music Melody Detection Approach

Long Short-Term Memory-Based Music Analysis System for Music Therapy

Music Similarity Detection Guided by Deep Learning Model

Multimodal Sentiment Analysis of Intangible Cultural Heritage Songs with Strengthened Audio Features-Guided Attention

Learning to Generate Music With Sentiment

Enhancing Music Mood Recognition with LLMs and Audio Signal Processing: A Multimodal Approach

Improve the application of reinforcement learning and multi‐modal information in music sentiment analysis

Boosting for Multi-Modal Music Emotion Classification.

A Joint Pitch Estimation and Voicing Detection Method for Melody Extraction

Exploring and Applying Audio-Based Sentiment Analysis in Music

Melody Extraction from Polyphonic Music by Deep Learning Approaches: A Review

A Music Cognition–Guided Framework for Multi-pitch Estimation

Machine Learning Framework for Audio-Based Content Evaluation using MFCC, Chroma, Spectral Contrast, and Temporal Feature Engineering

A data-driven approach to mid-level perceptual musical feature modeling