Abstract:Fusion of visual content with textual information is an effective way for both content-based and keyword-based image retrieval. However, the performance of visual & textual fusion is affected greatly by the data noise and redundancy in both text (such as surrounding text in HTML pages) and visual (such as intra-class diversity) aspects. This paper presents a manifold-based cross-media optimization scheme to achieve visual & textual fusion within a unified framework. Cross-Media manifold co-training mechanism between Keyword-based Metric Space and Vision-Based Metric Space is proposed creatively to infer a best dual-space fusion by minimizing manifold-based visual & textual energy criterion. We present the Isomorphic Manifold Learning to map the annotation affection in image visual space onto keyword semantic space by manifold shrinkage. We also demonstrate its correctness and convergence from mathematical perspective. The retrieval can be performed using both keyword or sample images respectively on Keyword-Based Metric Space and Vision-Based Metric Space, while the simple distance classifiers will satisfy. Two groups of experiments are conducted: The first group is carried on Corel 5000 image database to validate our effectiveness by comparing with state-of-the-art Generalized Manifold Ranking Based Image Retrieval and SVM. The second group is done over real-world Flickr dataset with over 6,000 images to testify our effectiveness in real-world application. The promising results show that our model attains a significant improvement over state-of-the-art algorithms.

Audio Retrieval Based on Manifold Ranking

Learning an Image Manifold for Retrieval

Audio retrieval by segment-based manifold-ranking

Multi-Manifold Ranking: Using Multiple Features for Better Image Retrieval.

Image-based 3D Model Retrieval Using Manifold Learning

Efficient Manifold Ranking for Image Retrieval

Audio Similarity Measure by Graph Modeling and Matching

EMR: A Scalable Graph-Based Ranking Model for Content-Based Image Retrieval

Integrated Manifold Ranking and Region Matching for Image Retrieval

A Method Based On General Model Template For Audio Retrieval

Cross-media Manifold Learning for Image Retrieval & Annotation

Audio Format Music Retrieval Based on Spectrogram Similarity

A multimedia retrieval framework based on semi-supervised ranking and relevance feedback.

Image Retrieval Algorithms Based on Manifold Learning

Multimodal Fusion Remote Sensing Image–Audio Retrieval

Audio Retrieval with Fast Relevance Feedback Based on Constrained Fuzzy Clustering and Stored Index Table.

Generalized Manifold-Ranking-Based Image Retrieval

Manifold-ranking Embedded Order Preserving Hashing for Image Semantic Retrieval

Estimating Visual Information From Audio Through Manifold Learning

Content Based Image Retrieval Using Manifold-Ranking of Blocks.

Inequivalent manifold ranking for content-based image retrieval