Abstract:The first step to apply deep learning techniques for symbolic music understanding is to transform musical pieces (mainly in MIDI format) into sequences of predefined tokens like note pitch, note velocity, and chords. Subsequently, the sequences are fed into a neural sequence model to accomplish specific tasks. Music sequences exhibit strong correlations between adjacent elements, making them prime candidates for N-gram techniques from Natural Language Processing (NLP). Consider classical piano music: specific melodies might recur throughout a piece, with subtle variations each time. In this paper, we propose a novel method, NG-Midiformer, for understanding symbolic music sequences that leverages the N-gram approach. Our method involves first processing music pieces into word-like sequences with our proposed unsupervised compoundation, followed by using our N-gram Transformer encoder, which can effectively incorporate N-gram information to enhance the primary encoder part for better understanding of music sequences. The pre-training process on large-scale music datasets enables the model to thoroughly learn the N-gram information contained within music sequences, and subsequently apply this information for making inferences during the fine-tuning stage. Experiment on various datasets demonstrate the effectiveness of our method and achieved state-of-the-art performance on a series of music understanding downstream tasks. The code and model weights will be released at <a class="link-external link-https" href="https://github.com/CinqueOrigin/NG-Midiformer" rel="external noopener nofollow">this https URL</a>.

Collective Annotation of Music from Multiple Semantic Categories

Semantic Music Annotation by Label-Specific Conditional Random Fields

Hierarchical Attentive Deep Neural Networks for Semantic Music Annotation Through Multiple Music Representations

Enriching Music Mood Annotation By Semantic Association Reasoning

A Semantic Context Model For Automatic Image Annotation

Music-to-Text Synaesthesia: Generating Descriptive Text from Music Recordings

Predicting High-level Music Semantics Using Social Tags via Ontology-based Reasoning.

Music Review Classification Enhanced by Semantic Information

MusicTM-Dataset for Joint Representation Learning among Sheet Music, Lyrics, and Musical Audio

Towards a new reading experience via semantic fusion of text and music.

Correlative Multi-Label Video Annotation.

A Survey of Foundation Models for Music Understanding

Music Genres Classification Using Text Categorization Method

A Unified Model for Zero-shot Music Source Separation, Transcription and Synthesis

Multimodel Music Emotion Recognition Using Unsupervised Deep Neural Networks

N-Gram Unsupervised Compoundation and Feature Injection for Better Symbolic Music Understanding

Multi-Modal Multi-Label Semantic Indexing Of Images Based On Hybrid Ensemble Learning

Improve the application of reinforcement learning and multi‐modal information in music sentiment analysis

Improving Music Genre Classification from Multi-Modal Properties of Music and Genre Correlations Perspective

Real-time Human-Music Emotional Interaction Based on Multimodal Analysis

Music Similarity Detection Guided by Deep Learning Model