Abstract:To go beyond the query-by-example paradigm in image retrieval, there is a need for semantic indexing of large image collections for intuitive text-based image search. Different models have been proposed to learn the dependencies between the visual content of an image set and the associated text captions, then allowing for the automatic creation of semantic indices for unannotated images. The task, however, remains unsolved. In this paper, we present three alternatives to learn a Probabilistic Latent Semantic Analysis model (PLSA) for annotated images, and evaluate their respective performance for automatic image indexing. Under the PLSA assumptions, an image is modeled as a mixture of latent aspects that generates both image features and text captions, and we investigate three ways to learn the mixture of aspects. We also propose a more discriminative image representation than the traditional Blob histogram, concatenating quantized local color information and quantized local texture descriptors. The first learning procedure of a PLSA model for annotated images is a standard EM algorithm, which implicitly assumes that the visual and the textual modalities can be treated equivalently. The other two models are based on an asymmetric PLSA learning, allowing to constrain the definition of the latent space on the visual or on the textual modality. We demonstrate that the textual modality is more appropriate to learn a semantically meaningful latent space, which translates into improved annotation performance. A comparison of our learning algorithms with respect to recent methods on a standard dataset is presented, and a detailed evaluation of the performance shows the validity of our framework.

Latent Topic Fusion-Based Cross-Media Image Semantic Annotation

Image Retrieval by Cross-Media Relevance Fusion.

Bridging the Semantic Gap Between Image Contents and Tags

A Semantic Context Model For Automatic Image Annotation

Crossmedia retrieval by learning rich semantic embeddings of multimedia

Modeling semantic aspects for cross-media image indexing

Topic models for image annotation and text illustration

Image Semantic Description and Automatic Semantic Annotation

A Probabilistic Semantic Model for Image Annotation and Multi-Modal Image Retrieval

A Feature-Word-topic Model for Image Annotation and Retrieval

Multi-Modal Multi-Label Semantic Indexing Of Images Based On Hybrid Ensemble Learning

A novel label-based multimodal topic model for social media analysis

Using Local Discriminant Topic To Improve Generative Model Based Image Annotation

Fusing Cross-Media for Topic Detection by Dense Keyword Groups

Automatic Image Annotation Using Social Group Latent Topic Mining and Multi-group Information Fusion

Semi-supervised topic modeling for image annotation.

Learning Multimodal Latent Attributes

Comment-Guided Semantics-Aware Image Aesthetics Assessment

Automatic annotation of weakly-tagged social images on flickr using latent topic discovery of multiple groups

Automatic Image Annotation Based on Topic-Based Smoothing

Learning Semantics for Image Annotation