Abstract:Fusion of visual content with textual information is an effective way for both content-based and keyword-based image retrieval. However, the performance of visual & textual fusion is affected greatly by the data noise and redundancy in both text (such as surrounding text in HTML pages) and visual (such as intra-class diversity) aspects. This paper presents a manifold-based cross-media optimization scheme to achieve visual & textual fusion within a unified framework. Cross-Media manifold co-training mechanism between Keyword-based Metric Space and Vision-Based Metric Space is proposed creatively to infer a best dual-space fusion by minimizing manifold-based visual & textual energy criterion. We present the Isomorphic Manifold Learning to map the annotation affection in image visual space onto keyword semantic space by manifold shrinkage. We also demonstrate its correctness and convergence from mathematical perspective. The retrieval can be performed using both keyword or sample images respectively on Keyword-Based Metric Space and Vision-Based Metric Space, while the simple distance classifiers will satisfy. Two groups of experiments are conducted: The first group is carried on Corel 5000 image database to validate our effectiveness by comparing with state-of-the-art Generalized Manifold Ranking Based Image Retrieval and SVM. The second group is done over real-world Flickr dataset with over 6,000 images to testify our effectiveness in real-world application. The promising results show that our model attains a significant improvement over state-of-the-art algorithms.

Content Based Image Retrieval Using Manifold-Ranking of Blocks.

Learning an Image Manifold for Retrieval

A Balanced Semi-Supervised Hashing Method for CBIR

Multistage and Multi-features Medical Image Retrieval System

Towards a unified approach to document similarity search using manifold-ranking of blocks

Integrated Manifold Ranking and Region Matching for Image Retrieval

Block-based similarity search on the web using manifold-ranking

Efficient Manifold Ranking for Image Retrieval

Multi-Manifold Ranking: Using Multiple Features for Better Image Retrieval.

Image-based 3D Model Retrieval Using Manifold Learning

EMR: A Scalable Graph-Based Ranking Model for Content-Based Image Retrieval

Generalized Manifold-Ranking-Based Image Retrieval

Online Learning to Rank for Content-Based Image Retrieval.

Inequivalent manifold ranking for content-based image retrieval

Retrieving Video Shots In Semantic Brain Imaging Space Using Manifold-Ranking

Deep Learning for Content-Based Image Retrieval: A Comprehensive Study

A comprehensive study on learning to rank for content-based image retrieval

A Novel Hierarchical Block Image Retrieval Scheme Based Invariant Features

Learning to Combine Ad-hoc Ranking Functions for Image Retrieval

Image retrieval using nonlinear manifold embedding

Cross-media Manifold Learning for Image Retrieval & Annotation