Temporal-aware Multimodal Feature Fusion for Sentiment Analysis

Feixiang Zhang,Ruotong Wang,Xiao Sun,Qi Li,Shulei Tang,Meng Wang,Yangyang Xu,Zhuoer Zhao
DOI: https://doi.org/10.1145/3606039.3613111
2023-10-29
Abstract:In this paper, we present a solution to the MuSe-Personalisation sub-challenge in the Multimodal Sentiment Analysis Challenge 2023. The task of MuSe-Personalisation aims to predict a time-continuous emotional value (i.e., arousal and valence) by using multimodal data. The MuSe-Personalisation sub-challenge faces the individual variations problem, resulting in poor generalization on unknown test sets. To solve the above problem, we first extract several informative visual features, and then propose a framework containing feature selection, feature learning and fusion strategy to discover the best combination of features for sentiment analysis. Finally, our method achieved the Top 1 performance in the MuSe-Personalisation sub-challenge, and the result in the combined CCC of physiological arousal and valence was 0.8681, outperforming the baseline system by a large margin (i.e., 10.42%) on the test set.
Computer Science
What problem does this paper attempt to address?