A Novel Multi-modal Integration and Propagation Model for Cross-Media Information Retrieval

Wanxia Lin,Tong Lu,Feng Su
DOI: https://doi.org/10.1007/978-3-642-27355-1_78
2012-01-01
Abstract:In this paper, we present a novel Probabilistic Latent Semantic Analysis-based (PLSA-based) aspect model and turn cross-media retrieval into two parts of multi-modal integration and correlation propagation. We first use multivariate Gaussian distributions to model continuous quantity in PLSA, avoiding information loss between feature-instance versus real-world matching. Multi-modal correlations are learned in an asymmetrical manner, giving a better control of the respective influence of each modality in the latent space. Then we propose a new propagation pattern to refine multi-modal correlations by efficiently taking the complementary from multi-modalities. Experimental results demonstrate that our method is accurate and robust for cross-media information retrieval.
What problem does this paper attempt to address?