Abstract:To go beyond the query-by-example paradigm in image retrieval, there is a need for semantic indexing of large image collections for intuitive text-based image search. Different models have been proposed to learn the dependencies between the visual content of an image set and the associated text captions, then allowing for the automatic creation of semantic indices for unannotated images. The task, however, remains unsolved. In this paper, we present three alternatives to learn a Probabilistic Latent Semantic Analysis model (PLSA) for annotated images, and evaluate their respective performance for automatic image indexing. Under the PLSA assumptions, an image is modeled as a mixture of latent aspects that generates both image features and text captions, and we investigate three ways to learn the mixture of aspects. We also propose a more discriminative image representation than the traditional Blob histogram, concatenating quantized local color information and quantized local texture descriptors. The first learning procedure of a PLSA model for annotated images is a standard EM algorithm, which implicitly assumes that the visual and the textual modalities can be treated equivalently. The other two models are based on an asymmetric PLSA learning, allowing to constrain the definition of the latent space on the visual or on the textual modality. We demonstrate that the textual modality is more appropriate to learn a semantically meaningful latent space, which translates into improved annotation performance. A comparison of our learning algorithms with respect to recent methods on a standard dataset is presented, and a detailed evaluation of the performance shows the validity of our framework.

A Probabilistic Semantic Model for Image Annotation and Multi-Modal Image Retrieval

A Semantic Context Model For Automatic Image Annotation

Multilateral Semantic Relations Modeling for Image Text Retrieval

Simultaneous Image Classification and Annotation Based on Probabilistic Model

A two-stage hybrid probabilistic topic model for refining image annotation

Multi-label image annotation based on multi-model

A Novel Semantics-Based Image Retrieval Method Using Similarity Measure of Multi-Level Semantics

Multimodal Image Retrieval Based on Annotation Keywords and Visual Content

Statistical Modeling and Conceptualization of Natural Images

Visual and textual fusion for semantically supervised region-based retrieval

A New Algorithm For Image Annotation Based On Markov Model

Automatic image annotation using semantic relevance

Automatic Image Annotation by Combining Aspects and Visual Semantics

Deep Image Annotation and Classification by Fusing Multi-Modal Semantic Topics

Efficient probabilistic image retrieval based on a mixed feature model

Image Retrieval Based on Multi-Concept Detector and Semantic Correlation.

Automatic Image Annotation Based-On Model Space

Multi-spectral Remote Sensing Image Retrieval Based on Semantic Extraction

Image Modeling with Combined Optimization Techniques for Image Semantic Annotation.

Modeling semantic aspects for cross-media image indexing

Towards Multi-Semantic Image Annotation with Graph Regularized Exclusive Group Lasso