Abstract:Controllable music generation promotes the interaction between humans and composition systems by projecting the users' intent on their desired music. The challenge of introducing controllability is an increasingly important issue in the symbolic music generation field. When building controllable generative popular multi-instrument music systems, two main challenges typically present themselves, namely weak controllability and poor music quality. To address these issues, we first propose spatiotemporal features as powerful and fine-grained controls to enhance the controllability of the generative model. In addition, an efficient music representation called REMI_Track is designed to convert multitrack music into multiple parallel music sequences and shorten the sequence length of each track with Byte Pair Encoding (BPE) techniques. Subsequently, we release BandControlNet, a conditional model based on parallel Transformers, to tackle the multiple music sequences and generate high-quality music samples that are conditioned to the given spatiotemporal control features. More concretely, the two specially designed modules of BandControlNet, namely structure-enhanced self-attention (SE-SA) and Cross-Track Transformer (CTT), are utilized to strengthen the resulting musical structure and inter-track harmony modeling respectively. Experimental results tested on two popular music datasets of different lengths demonstrate that the proposed BandControlNet outperforms other conditional music generation models on most objective metrics in terms of fidelity and inference speed and shows great robustness in generating long music samples. The subjective evaluations show BandControlNet trained on short datasets can generate music with comparable quality to state-of-the-art models, while outperforming them significantly using longer datasets.

All-In-One Metrical And Functional Structure Analysis With Neighborhood Attentions on Demixed Audio

Learning Hierarchical Metrical Structure Beyond Measures

Symbolic Music Structure Analysis with Graph Representations and Changepoint Detection Methods

InstructME: An Instruction Guided Music Edit And Remix Framework with Latent Diffusion Models

An Analytical Framework of Tonal and Rhythmic Hierarchy in Natural Music Using the Multivariate Temporal Response Function

Small Tunes Transformer: Exploring Macro & Micro-Level Hierarchies for Skeleton-Conditioned Melody Generation

Hierarchical Regulated Iterative Network for Joint Task of Music Detection and Music Relative Loudness Estimation.

Global Structure-Aware Drum Transcription Based on Self-Attention Mechanisms

BandControlNet: Parallel Transformers-based Steerable Popular Music Generation with Fine-Grained Spatiotemporal Features

Self-Similarity-Based and Novelty-based loss for music structure analysis

InstructME: an Instruction Guided Music Edit Framework with Latent Diffusion Models

A Unified Model for Zero-shot Music Source Separation, Transcription and Synthesis

Modeling Musical Structure with Artificial Neural Networks

Improved Architecture for High-resolution Piano Transcription to Efficiently Capture Acoustic Characteristics of Music Signals

Self-Supervised Learning of Multi-level Audio Representations for Music Segmentation

Unlocking Potential in Pre-Trained Music Language Models for Versatile Multi-Track Music Arrangement

Convolutive Block-Matching Segmentation Algorithm with Application to Music Structure Analysis

HANME: Hierarchical Attention Network for Singing Melody Extraction

Hierarchical Recurrent Neural Networks for Conditional Melody Generation with Long-term Structure

A Computational Analysis of Real-World DJ Mixes using Mix-To-Track Subsequence Alignment

Hierarchical Attentive Deep Neural Networks for Semantic Music Annotation Through Multiple Music Representations