Abstract:To go beyond the query-by-example paradigm in image retrieval, there is a need for semantic indexing of large image collections for intuitive text-based image search. Different models have been proposed to learn the dependencies between the visual content of an image set and the associated text captions, then allowing for the automatic creation of semantic indices for unannotated images. The task, however, remains unsolved. In this paper, we present three alternatives to learn a Probabilistic Latent Semantic Analysis model (PLSA) for annotated images, and evaluate their respective performance for automatic image indexing. Under the PLSA assumptions, an image is modeled as a mixture of latent aspects that generates both image features and text captions, and we investigate three ways to learn the mixture of aspects. We also propose a more discriminative image representation than the traditional Blob histogram, concatenating quantized local color information and quantized local texture descriptors. The first learning procedure of a PLSA model for annotated images is a standard EM algorithm, which implicitly assumes that the visual and the textual modalities can be treated equivalently. The other two models are based on an asymmetric PLSA learning, allowing to constrain the definition of the latent space on the visual or on the textual modality. We demonstrate that the textual modality is more appropriate to learn a semantically meaningful latent space, which translates into improved annotation performance. A comparison of our learning algorithms with respect to recent methods on a standard dataset is presented, and a detailed evaluation of the performance shows the validity of our framework.

How Does Latent Semantic Analysis Work? A Visualisation Approach

The Theory and Application of Latent Semantic Analysis

Visualizing and Understanding Neural Models in NLP

Text Information Retrieval Based on Latent Semantic Analysis

Latent Space Cartography: Visual Analysis of Vector Space Embeddings

On the connections between explicit semantic analysis and latent semantic analysis.

Efficient Probabilistic Latent Semantic Analysis with Sparsity Control

Latent Semantic Scaling: A Semisupervised Text Analysis Technique for New Domains and Languages

The interpretation of dream meaning: Resolving ambiguity using Latent Semantic Analysis in a small corpus of text

Comparative summarization via Latent Semantic Analysis

On the Semantics of LM Latent Space: A Vocabulary-defined Approach

Semantic Lens: a Visualization and Inspection Tool for Multi-volume Data

Explaining Deep Convolutional Neural Networks via Latent Visual-Semantic Filter Attention

The Latent Relation Mapping Engine: Algorithm and Experiments

Towards Visual Semantics

Latent Concept-based Explanation of NLP Models

Semantic Visualization for Short Texts with Word Embeddings

Modeling semantic aspects for cross-media image indexing

ExplainLFS: Explaining neural architectures for similarity learning from local perturbations in the latent feature space

Latent Space Translation via Semantic Alignment

LDAExplore: Visualizing Topic Models Generated Using Latent Dirichlet Allocation