Automated Music Video Generation Using Web Image Resource

Rui Cai,Lei Zhang,Feng Jing,Wei Lai,Wei-Ying Ma

DOI: https://doi.org/10.1109/ICASSP.2007.366341

2007-01-01

Abstract:In this paper, we proposed a novel prototype of automated music video generation using web image resource. In this prototype, the salient words/phrases of a song's lyrics are first automatically extracted and then used as queries to retrieve related high-quality images from web search engines. To guarantee the coherence among the chosen images' visual representation and the music song, the returned images are further re-ranked and filtered based on their content characteristics such as color, face, landscape, as well as the song's mood type. Finally, those selected images are concatenated to generate a music video using the Photo2Video technique, based on the rhythm information of the music. Preliminary evaluations of the proposed prototype have shown promising results.