Learning based relevance feedback in image retrieval

Yong Rui,Thomas Huang

DOI: https://doi.org/10.1142/9789812776952_0007

2002-01-01

Abstract:The following sections are included: Introduction Concepts and Notations Related Work The Proposed Approach Experiments, Results and Evaluations Discussions and Conclusions Acknowledgment References

What problem does this paper attempt to address?

Learning A Semantic Space from User'S Relevance Feedback for Image Retrieval

XF He,O King,WY Ma,MJ Li,HJ Zhang

DOI: https://doi.org/10.1109/tcsvt.2002.808087

2002-01-01

Abstract:As current methods for content-based retrieval are incapable of capturing the semantics of images, we experiment with using spectral methods to infer a semantic space from user's relevance feedback, so that our system will gradually improve its retrieval performance through accumulated user interactions. In addition to the long-term learning process, we also model the traditional approaches to query refinement using relevance feedback as a short-term learning process. The proposed short- and long-term learning frameworks have been integrated into an image retrieval system. Experimental results on a large collection of images have shown the effectiveness and robustness of our proposed algorithms.
Image retrieval based on incremental subspace learning

Ke Lu,Xiaofei He

DOI: https://doi.org/10.1016/j.patcog.2005.05.005

IF: 8

2005-01-01

Pattern Recognition

Abstract:Many problems in information processing involve some form of dimensionality reduction, such as face recognition, image/text retrieval, data visualization, etc. The typical linear dimensionality reduction algorithms include principal component analysis (PCA), random projection, locality-preserving projection (LPP), etc. These techniques are generally unsupervised which allows them to model data in the absence of labels or categories. In this paper, we propose a semi-supervised subspace learning algorithm for image retrieval. In relevance feedback-driven image retrieval system, the user-provided information can be used to better describe the intrinsic semantic relationships between images. Our algorithm is fundamentally based on LPP which can incorporate user's relevance feedbacks. As the user's feedbacks are accumulated, we can ultimately obtain a semantic subspace in which different semantic classes can be best separated and the retrieval performance can be enhanced. We compared our proposed algorithm to PCA and the standard LPP. Experimental results on a large collection of images have shown the effectiveness and efficiency of our proposed algorithm.
Learning an Image Manifold for Retrieval

Xiaofei He,Wei-Ying Ma,HongJiang Zhang

DOI: https://doi.org/10.1145/1027527.1027532

2004-01-01

Abstract:We consider the problem of learning a mapping function from low-level feature space to high-level semantic space. Under the assumption that the data lie on a submanifold embedded in a high dimensional Euclidean space, we propose a relevance feedback scheme which is naturally conducted only on the image manifold in question rather than the total ambient space. While images are typically represented by feature vectors in Rn, the natural distance is often different from the distance induced by the ambient space Rn. The geodesic distances on manifold are used to measure the similarities between images. However, when the number of data points is small, it is hard to discover the intrinsic manifold structure. Based on user interactions in a relevance feedback driven query-by-example system, the intrinsic similarities between images can be accurately estimated. We then develop an algorithmic framework to approximate the optimal mapping function by a Radial Basis Function (RBF) neural network. The semantics of a new image can be inferred by the RBF neural network. Experimental results show that our approach is effective in improving the performance of content-based image retrieval systems.
An Effective and Simple Relevance Feedback Algorithm for Image Retrieval

N Yang,XY Xue

DOI: https://doi.org/10.1117/12.476246

2003-01-01

Abstract:Relevance feedback in content-based image retrieval has been an active research focus for many years. It uses user-labeled information to re-adjust the measurement of similarity between images to get the improved retrieval results. In this paper we propose a simple and effective approach for image relevance feedback, which uses both positive and negative examples labeled by users to refine the query and update the distance measurement dynamically. Our method not only has a very low complexity but also adapts well to the changes of user's retrieval interests. Experimental results on a database of 7,000 images represented by MPEG-7 color and texture descriptors show the efficiency of our algorithm compared with other two existing algorithms.
Online Metric Learning for Relevance Feedback in E-Commerce Image Retrieval Applications

Hong Gu,Guangzhou Zhao,Jun Qiu

DOI: https://doi.org/10.1016/s1007-0214(11)70055-4

2011-01-01

Abstract:Relevance feedback plays a key role in multiple feature-based image retrieval applications. This paper describes an online metric learning approach for a set of ranking functions. In the feedback round, the most relevant and most nonrelevant images related to the target image are selected to construct a relative comparison triplet. The weighting parameters of the multiple ranking functions are updated by minimizing a quadratic objective function constrained by the triplet. The approach unifies the learning algorithm for the most commonly used ranking functions. Thus, multiple features with their own ranking function can easily be employed in the ranking module without feature reconstruction. The method is computationally inexpensive and appropriate for large-scale e-commerce image retrieval applications. Customized ranking functions are well supported. Practically, simplified ranking functions yield better results when the number of query rounds is relatively small. Experiments with an image dataset from a real e-commerce platform show the superiority of the proposed approach.
A Novel Relevance Feedback Technique in Image Retrieval

Yong Rui,Thomas S. Huang

DOI: https://doi.org/10.1145/319878.319896

1999-01-01

Abstract:The relevance feedback based approach to image retrieval has been an active research direction in the past few years. Many parameter estimation techniques have been proposed for relevance feedback. However, most of them are either based on ad-hoc heuristics or only partial solutions. In this paper, we introduce the first technique that not only has a solid theoretical framework but also takes into account the multi-level image content model. This technique formulates a vigorous optimization problem. By using Lagrange multipliers, we have derived the explicit optimal solutions for both the query vectors and the weights associated with the two-level image model. Experimental results on realworld image collections have shown the effectiveness and robustness of our proposed algorithm.
Enhancing Relevance Feedback in Image Retrieval Using Unlabeled Data

Zhi-Hua Zhou,Ke-Jia Chen,Hong-Bin Dai

DOI: https://doi.org/10.1145/1148020.1148023

2006-01-01

Abstract:Relevance feedback is an effective scheme bridging the gap between high-level semantics and low-level features in content-based image retrieval (CBIR). In contrast to previous methods which rely on labeled images provided by the user, this article attempts to enhance the performance of relevance feedback by exploiting unlabeled images existing in the database. Concretely, this article integrates the merits of semisupervised learning and active learning into the relevance feedback process. In detail, in each round of relevance feedback two simple learners are trained from the labeled data, that is, images from user query and user feedback. Each learner then labels some unlabeled images in the database for the other learner. After retraining with the additional labeled data, the learners reclassify the images in the database and then their classifications are merged. Images judged to be positive with high confidence are returned as the retrieval result, while those judged with low confidence are put into the pool which is used in the next round of relevance feedback. Experiments show that using semisupervised learning and active learning simultaneously in CBIR is beneficial, and the proposed method achieves better performance than some existing methods.
Relevance Feedback in Content-Based Image Retrieval: Bayesian Framework, Feature Subspaces, and Progressive Learning

Z Su,HJ Zhang,S Li,SP Ma

DOI: https://doi.org/10.1109/tip.2003.815254

IF: 10.6

2003-01-01

IEEE Transactions on Image Processing

Abstract:Research has been devoted in the past few years to relevance feedback as an effective solution to improve performance of content-based image retrieval (CBIR). In this paper, we propose a new feedback approach with progressive learning capability combined with a novel method for the feature subspace extraction. The proposed approach is based on a Bayesian classifier and treats positive and negative feedback examples with different strategies. Positive examples are used to estimate a Gaussian distribution that represents the desired images for a given query; while the negative examples are used to modify the ranking of the retrieved candidates. In addition, feature subspace is extracted and updated during the feedback process using a principal component analysis (PCA) technique and based on user's feedback. That is, in addition to reducing the dimensionality of feature spaces, a proper subspace for each type of features is obtained in the feedback process to further improve the retrieval accuracy. Experiments demonstrate that the proposed method increases the retrieval speed, reduces the required memory and improves the retrieval accuracy significantly.
Relevance Feedback: a Power Tool for Interactive Content-Based Image Retrieval

Y Rui,TS Huang,M Ortega,S Mehrotra

DOI: https://doi.org/10.1109/76.718510

IF: 5.859

1998-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Content-based image retrieval (CBIR) has become one of the most active research areas in the past few years. Many visual feature representations have been explored and many systems built. While these research efforts establish the basis of CBIR, the usefulness of the proposed approaches is limited. Specifically, these efforts have relatively ignored two distinct characteristics of CBIR systems: (1) the gap between high-level concepts and low-level features, and (2) the subjectivity of human perception of visual content. This paper proposes a relevance feedback based interactive retrieval approach, which effectively takes into account the above two characteristics in CBIR. During the retrieval process, the user's high-level query and perception subjectivity are captured by dynamically updated weights based on the user's feedback. The experimental results over more than 70000 images show that the proposed approach greatly reduces the user's effort of composing a query, and captures the user's information need more precisely.
Relevance Feedback Techniques In Interactive Content-Based Image Retrieval

Y Rui,Ts Huang,S Mehrotra

DOI: https://doi.org/10.1117/12.298455

1997-01-01

Abstract:Content-Based Image Retrieval (CBIR) has become one of the most active research areas in the past few years. Many visual feature representations have been explored and many systems built. While these research efforts establish the basis of CBIR, the usefulness of the proposed approaches is limited. Specifically, these efforts have relatively ignored two distinct characteristics of CBIR systems: (1) the gap between high level concepts and low level features; (2) subjectivity of human perception of visual content.This paper proposes a relevance feedback based interactive retrieval approach, which effectively takes into account the above two characteristics in CBIR. During the retrieval process, the user's high level query and perception subjectivity are captured by dynamically updated weights based on the user's relevance feedback. The experimental results show that the proposed approach greatly reduces the user's effort of composing a query and captures the user's information need more precisely.
An active feedback framework for image retrieval

Tao Qin,Xu-Dong Zhang,Tie-Yan Liu,De-Sheng Wang,Wei-Ying Ma,Hong-Jiang Zhang

DOI: https://doi.org/10.1016/j.patrec.2007.11.015

IF: 4.757

2008-01-01

Pattern Recognition Letters

Abstract:In recent years, relevance feedback has been studied extensively as a way to improve performance of content-based image retrieval (CBIR). Since users are usually unwilling to provide much feedback, the insufficiency of training samples limits the success of relevance feedback. In this paper, we propose two strategies to tackle this problem: (i) to make relevance feedback more informative by presenting representative images for users to label; (ii) to make use of unlabeled data in the training process. As a result, an active feedback framework is proposed, consisting of two components, representative image selection and label propagation. For practical implementation of this framework, we develop two coupled algorithms corresponding to the two components, namely, overlapped subspace clustering and multi-subspace label propagation. Experimental results on a very large-scale image collection demonstrated the high effectiveness of the proposed active feedback framework.
Support Vector Machine Based Relevance Feedback Algorithm in Image Retrieval

张磊,林福宗,张钹

DOI: https://doi.org/10.3321/j.issn:1000-0054.2002.01.024

2002-01-01

Abstract:Relevance feedback technique has been an important approach in image retrieval. A novel relevance feedback algorithm is presented based on Support Vector Machine learning in content based image retrieval system. During the retrieval process, users can mark positive sample images similar to the query image. Then the algorithm constructs a SVM classifier, which can be used to find more relevance images in the whole image database. Experiments were carried out on a large size database of 9918 images. It shows that more images relevant to the query can be found efficiently by the interactive learning and retrieval process. It also shows the generalization ability of SVM under the conditions of limited training samples.
Self-Adaptive Relevance Feedback Based on Multi-Level Image Content Analysis

YY Gao,YJ Zhang,Y Fu

DOI: https://doi.org/10.1117/12.410955

2001-01-01

Abstract:In current content-based image retrieval systems, it is generally accepted that obtaining high-level image features is a key to improve the querying. Among the related techniques, relevance feedback has become a hot research aspect because it combines the information from the user to refine the querying results. In practice, many methods have been proposed to achieve the goal of relevance feedback. In this paper, a new scheme for relevance feedback is proposed. Unlike previous methods for relevance feedback, our scheme provides a self-adaptive operation. First, based on multi-level image content analysis, the relevant images from the user could be automatically analyzed in different levels and the querying could be modified in terms of different analysis results. Secondly, to make it more convenient to the user, the procedure of relevance feedback could be led with memory or without memory. To test the performance of the proposed method, a practical semantic-based image retrieval system has been established, and the querying results gained by our self-adaptive relevance feedback are given.
Relevance Feedback For Keyword And Visual Feature-Based Image Retrieval

F Jing,Mj Li,Hj Zhang,B Zhang

DOI: https://doi.org/10.1007/978-3-540-27814-6_52

2004-01-01

Abstract:In this paper, a relevance feedback scheme for both keyword and visual feature-based image retrieval is proposed. For each keyword, a statistical model is trained offline based on visual features of a small set of manually labeled images and used to propagate the keyword to other unlabeled ones. Besides the offline model, another model is constructed online using the user provided positive and negative images as training set. Support vector machines (SVMs) in the binary setting are adopted as both offline and online models. To effectively combine the two models, a multi-model query refinement algorithm is introduced. Furthermore, an entropy-based active learning strategy is proposed to improve the efficiency of relevance feedback process. Experimental results on a database of 10,000 general-purpose images demonstrate the effectiveness of the proposed relevance feedback scheme.
Support Vector Machine Learning for Image Retrieval

Z Lei,FZ Lin,Z Bo

DOI: https://doi.org/10.1109/icip.2001.958595

2001-01-01

Abstract:A novel method of relevance feedback is presented based on support vector machine learning in the content-based image retrieval system. A SVM classifier can be learned from training data of relevance images and irrelevance images marked by users. Using the classifier, the system can retrieve more images relevant to the query in the database efficiently. Experiments were carried out on a large-size database of 9918 images. It shows that the interactive learning and retrieval process can find correct images increasingly. It also shows the generalization ability of SVM under the condition of limited training samples.
A Survey on Relevance Feedback Techniques in Content-based Medical Image Retrieval

Ye SHEN,Shun-Ren XIA,Min-Dan LI

DOI: https://doi.org/10.3969/j.issn.0258-8021.2009.01.023

2009-01-01

Abstract:Efficiently and accurately searching for similar medical images in databases is a challenging task. Relevance feedback (RF) ,a effective approach to bridge "semantic gap" and boost image retrieval, has become a key part of content-based medical image retrieval (CBMIR) . RF algorithms can be categorized into three classes according to the retrieval models adopted in them. They are distance-based model, probabilistic model and machine learning based model. Following this categorization, representative algorithms were analyzed and evaluated in this paper with an emphasis on the machine learning algorithms. At last, some promising research directions about RF in CBMIR were discussed.
Relevance feedback: A power tool for interactive content-based image retrieval.

P Ambika,J Abdul Samath,S Aksoy,RM Haralick,FA Cheikh,M Gabbouj,P Ambika,JA Samath,CH Chang,CC Hsu,IJ Cox,ML Miller,TP Minka,TV Papathomas,PN Yianilos,J Peng,B Bhanu,S Qing,Y Rui,T Huang,Y Rui,TS Huang,M Ortega,S Mehrotra

2000-01-01

Research Journal of Information Technology

Abstract:Description of content as well as semantics is important in Content Based Image Retrieval. Even though the image gets interpreted semantically, the retrieval accuracy of CBIR systems is often low. Many CBIR systems still rely on text retrieval technologies on human labeled keywords. Identifying and Learning the interest point of humans are critical components in Image reclamation. This manuscript advises a Neural Network scaffold that incorporates unsupervised learning in to query refinement process. In order to indentify the user interest point we have used this as a relevance feedback approach which easily maps stumpy characteristics with user’s lofty concept. Experimental results illustrate the effectiveness of this approach.
Relevance Feedback Learning with Feature Selection in Region-Based Image Retrieval

W Jiang,GH Er,QH Dai,L Zhong,Y Hou

DOI: https://doi.org/10.1109/icassp.2005.1415453

2005-01-01

Abstract:Region-based image retrieval and relevance feedback are two important methods to bridge the gap between the low-level visual features and the high-level semantic concepts in content-based image retrieval. In this paper, we address the issue of introducing the relevance feedback mechanism into the region-based image retrieval with online feature selection during each feedback round. Our contribution has two folds. (1) A novel region-based image representation is proposed. Based on a generative model, a fuzzy code-book is extracted from the original region-based features, which represents the images in a uniform real-value feature space. (2) A feature selection criterion is developed and an effective relevance feedback algorithm is implemented in a boosting manner to simultaneously select the optimal features from the fuzzy codebook, and generate a strong ensemble classifier over the selected features. Experimental results show that the proposed scheme can substantially improve the retrieval performance of the region-based image retrieval system.
Towards Optimal Query Design For Relevance Feedback In Image Retrieval

Jingyu Cui,Changshui Zhang

DOI: https://doi.org/10.1109/ICASSP.2008.4517837

2008-01-01

ICASSP

Abstract:We analyze the sub-optimality of traditional greedy active learning based relevance feedback methods in image retrieval, and propose a novel active learning approach to query labels of multiple images together, which minimize the needed round of feedbacks and achieve satisfactory result in a near optimal manner. Our experiments on real image retrieval demonstrate that our solution can yield comparable precession/recall rate by significantly less relevance feedbacks.
&Lt;title>information Embedding Based on User's Relevance Feedback for Image Retrieval</title>

Catherine S. Lee,Wei‐Ying Ma,Hong-Jiang Zhang

DOI: https://doi.org/10.1117/12.360434

1999-01-01

Abstract:An image retrieval system based on an information embedding scheme is proposed. Using relevance feedback, the system gradually embeds correlations between images from a high- level semantic perspective. The system starts with low-level image features and acquires knowledge from users to correlate different images in the database. Through the selection of positive and negative examples based on a given query, the semantic relationships between images are captured and embedded into the system by splitting/merging image clusters and updating the correlation matrix. Image retrieval is then based on the resulting image clusters and the correlation matrix obtained through relevance feedback.

Learning based relevance feedback in image retrieval

Learning A Semantic Space from User'S Relevance Feedback for Image Retrieval

Image retrieval based on incremental subspace learning

Learning an Image Manifold for Retrieval

An Effective and Simple Relevance Feedback Algorithm for Image Retrieval

Online Metric Learning for Relevance Feedback in E-Commerce Image Retrieval Applications

A Novel Relevance Feedback Technique in Image Retrieval

Enhancing Relevance Feedback in Image Retrieval Using Unlabeled Data

Relevance Feedback in Content-Based Image Retrieval: Bayesian Framework, Feature Subspaces, and Progressive Learning

Relevance Feedback: a Power Tool for Interactive Content-Based Image Retrieval

Relevance Feedback Techniques In Interactive Content-Based Image Retrieval

An active feedback framework for image retrieval

Support Vector Machine Based Relevance Feedback Algorithm in Image Retrieval

Self-Adaptive Relevance Feedback Based on Multi-Level Image Content Analysis

Relevance Feedback For Keyword And Visual Feature-Based Image Retrieval

Support Vector Machine Learning for Image Retrieval

A Survey on Relevance Feedback Techniques in Content-based Medical Image Retrieval

Relevance feedback: A power tool for interactive content-based image retrieval.

Relevance Feedback Learning with Feature Selection in Region-Based Image Retrieval

Towards Optimal Query Design For Relevance Feedback In Image Retrieval

&Lt;title&gt;information Embedding Based on User's Relevance Feedback for Image Retrieval&lt;/title&gt;

&Lt;title>information Embedding Based on User's Relevance Feedback for Image Retrieval</title>