A Comprehensive Survey for Evaluation Methodologies of AI-Generated Music

Zeyu Xiong,Weitao Wang,Jing Yu,Yue Lin,Ziyan Wang

2023-08-26

Abstract:In recent years, AI-generated music has made significant progress, with several models performing well in multimodal and complex musical genres and scenes. While objective metrics can be used to evaluate generative music, they often lack interpretability for musical evaluation. Therefore, researchers often resort to subjective user studies to assess the quality of the generated works, which can be resource-intensive and less reproducible than objective metrics. This study aims to comprehensively evaluate the subjective, objective, and combined methodologies for assessing AI-generated music, highlighting the advantages and disadvantages of each approach. Ultimately, this study provides a valuable reference for unifying generative AI in the field of music evaluation.

Sound,Artificial Intelligence,Human-Computer Interaction,Audio and Speech Processing

What problem does this paper attempt to address?

The paper aims to comprehensively evaluate the methods for assessing AI-generated music, including subjective, objective, and combined approaches, and highlights the advantages and disadvantages of each method. Specifically:

1. **Research Background**: In recent years, AI-generated music has made significant progress. Although existing evaluation methods (such as objective metrics and subjective user studies) each have their strengths and weaknesses, there is a lack of unified standards.

2. **Main Contributions**:
- Provides a classification reference scheme for evaluation in the field of AI music creation.
- Offers reference value for the unification of evaluation standards for music generation models.

3. **Evaluation Methods**:
- **Subjective Evaluation**: Relies on listener satisfaction, conducted through music listening tests (such as the music Turing test) and visual analysis.
- **Objective Evaluation**: Uses computational techniques to analyze music quality, such as model-based metrics (training loss, accuracy, etc.) and music domain metrics (pitch correlation, rhythm variation, etc.).
- **Combined Evaluation**: Combines subjective and objective methods to compensate for their respective shortcomings.

4. **Future Directions**:
- Establish standardized evaluation systems.
- Bridge the gap between subjective and objective evaluations.
- Enhance the interpretability of objective metrics.
- Effectively evaluate creativity.

Through the above work, the paper provides a systematic review of AI music generation evaluation and points out the challenges and development directions in this field.