Social-Guided Representation Learning for Images via Deep Heterogeneous Hypergraph Embedding.

Yunfei Chu,Chunyan Feng,Caili Guo
DOI: https://doi.org/10.1109/ICME.2018.8486506
2018-01-01
Abstract:Representation learning for images is widely recognized as critical to the performance of end tasks such as image classification and cross-modal retrieval. However, most existing methods extract features only from visual content, far from adequate in interpreting semantics latent in images. For social images, there also exists rich social context information, e.g. owners, tags and groups, which provides cues for interpreting semantics. In this paper, we propose a representation learning framework via deep heterogeneous hypergraph embedding (DHHE), considering both visual content and social contexts. In particular, images and their contexts are first represented as a heterogeneous hypergraph, which is then embedded into a low-dimensional space. To incorporate visual information and generalize for unseen images, we learn the mapping from visual content to the semantic space. We conduct experiments with the tasks of classification, cross-modal retrieval and recommendation, which demonstrates the effectiveness of our approach and the merits of social guidance.
What problem does this paper attempt to address?