Pop Music Generation

Hongyuan Zhu,Qi Liu,Nicholas Jing Yuan,Kun Zhang,Guang Zhou,Enhong Chen

DOI: https://doi.org/10.1145/3374915

IF: 4.157

2020-08-21

ACM Transactions on Knowledge Discovery from Data

Abstract:Music plays an important role in our daily life. With the development of deep learning and modern generation techniques, researchers have done plenty of works on automatic music generation. However, due to the special requirements of both melody and arrangement, most of these methods have limitations when applying to multi-track music generation. Some critical factors related to the quality of music are not well addressed, such as chord progression, rhythm pattern, and musical style. In order to tackle the problems and ensure the harmony of multi-track music, in this article, we propose an end-to-end melody and arrangement generation framework to generate a melody track with several accompany tracks played by some different instruments. To be specific, we first develop a novel Chord based Rhythm and Melody Cross-Generation Model to generate melody with a chord progression. Then, we propose a Multi-Instrument Co-Arrangement Model based on multi-task learning for multi-track music arrangement. Furthermore, to control the musical style of arrangement, we design a Multi-Style Multi-Instrument Co-Arrangement Model to learn the musical style with adversarial training. Therefore, we can not only maintain the harmony of the generated music but also control the musical style for better utilization. Extensive experiments on a real-world dataset demonstrate the superiority and effectiveness of our proposed models.

computer science, information systems, software engineering

What problem does this paper attempt to address?

The paper attempts to address several key challenges in automatic music generation, particularly for multi-track arrangement in pop music. Specifically, the authors point out the limitations of current automatic music generation methods in handling melody and arrangement, especially when generating multi-track music. These issues include:

1. **Chord Progression**: Most existing methods do not consider chord progression when generating music, whereas chord progression is crucial for guiding the development of the melody.
2. **Rhythm Patterns**: Existing methods typically generate music note by note, ignoring rhythm patterns, which makes the generated music lack a sense of structure.
3. **Multi-track Arrangement**: Existing methods mainly focus on single-track music generation, while actual music works are usually the result of multi-instrument collaboration, requiring consideration of harmony between different instruments.
4. **Music Style Control**: Existing generation methods often do not consider music style, resulting in generated music that lacks diversity, flexibility, and authenticity.

To address these issues, the authors propose an end-to-end melody and arrangement generation framework aimed at generating melodies with multiple accompaniment tracks, ensuring harmony in chord, rhythm, and instrument arrangement, while being able to control the music style. The specific technical contributions include:

- **Chord-based Rhythm and Melody Cross-Generation Model (CRMCG)**: Utilizes chord progression to generate melody and rhythm, ensuring the generated music has structure and harmony.
- **Multi-Instrument Collaborative Arrangement Model (MICA)**: Generates multi-track music through multi-task learning, ensuring harmony between different instruments.
- **Multi-Style Multi-Instrument Collaborative Arrangement Model (MSMICA)**: Builds on MICA by introducing adversarial training and multi-task learning to control the style of the generated music, making it more diverse and realistic.

Through these techniques, the authors aim to generate high-quality, diverse multi-track pop music. Experimental results show that the proposed models outperform baseline methods in both single-track and multi-track music generation tasks.