Can Machines Generate Personalized Music A Hybrid Favorite-aware Method for User Preference Music Transfer

Zhejing Hu,Yan Liu,Gong Chen,Yongxu Liu

DOI: https://doi.org/10.1109/tmm.2022.3146002

IF: 7.3

2022-01-01

IEEE Transactions on Multimedia

Abstract:User preference music transfer (UPMT) is a new problem in music style transfer that can be applied to many scenarios but remains understudied. Transferring an arbitrary song to fit a users preferences increases musical diversity and improves user engagement, which can greatly benefit individuals mental health. Most music style transfer approaches rely on data-driven methods. In general, however, constructing a large training dataset is challenging because users can rarely provide enough of their favorite songs. To address this problem, this paper proposes a novel hybrid method called User Preference Transformer (UP-Transformer) which uses prior knowledge of only one piece of a users favorite music. Based on the distribution of music events in the provided music, we propose a new favorite-aware loss function to fine-tune the Transformer-based model. Two steps are proposed in the transfer phase to achieve UPMT based on the extracted music pattern in a users favorite music. Additionally, to alleviate the problem of evaluating melodic similarity in music style transfer, we propose a new concept called pattern similarity (PS) to measure the similarity between two pieces of music. Statistical tests indicate that the results of PS are consistent with the similarity score in a qualitative experiment. Furthermore, experimental results on subjects show that the transferred music achieves better performance in musicality, similarity, and user preferences.

computer science, information systems,telecommunications, software engineering