Abstract:BACKGROUND: The majority of current medical CBIR systems perform retrieval based only on "imaging signatures" generated by extracting pixel-level quantitative features, and only rarely has a feedback mechanism been incorporated to improve retrieval performance. In addition, current medical CBIR approaches do not routinely incorporate semantic terms that model the user's high-level expectations, and this can limit CBIR performance.METHOD: We propose a retrieval framework that exploits a hybrid feature space (HFS) that is built by integrating low-level image features and high-level semantic terms, through rounds of relevance feedback (RF) and performs similarity-based retrieval to support semi-automatic image interpretation. The novelty of the proposed system is that it can impute the semantic features of the query image by reformulating the query vector representation in the HFS via user feedback. We implemented our framework as a prototype that performs the retrieval over a database of 811 radiographic images that contains 69 unique types of bone tumors.RESULTS: We evaluated the system performance by conducting independent reading sessions with two subspecialist musculoskeletal radiologists. For the test set, the proposed retrieval system at fourth RF iteration of the sessions conducted with both the radiologists achieved mean average precision (MAP) value ∼0.90 where the initial MAP with baseline CBIR was 0.20. In addition, we also achieved high prediction accuracy (>0.8) for the majority of the semantic features automatically predicted by the system.CONCLUSION: Our proposed framework addresses some limitations of existing CBIR systems by incorporating user feedback and simultaneously predicting the semantic features of the query image. This obviates the need for the user to provide those terms and makes CBIR search more efficient for inexperience users/trainees. Encouraging results achieved in the current study highlight possible new directions in radiological image interpretation employing semantic CBIR combined with relevance feedback of visual similarity.

Automated Feedback Extraction for Medical Imaging Retrieval

Multistage and Multi-features Medical Image Retrieval System

A Survey on Relevance Feedback Techniques in Content-based Medical Image Retrieval

A Retrieval System For 3d Multi-Phase Contrast-Enhanced Ct Images Of Focal Liver Lesions Based On Combined Bags Of Visual Words And Texture Words

Relevance feedback for enhancing content based image retrieval and automatic prediction of semantic image features: Application to bone tumor radiographs

Mammogram Retrieval: Image Selection Strategy of Relevance Feedback for Locating Similar Lesions

Relevance Feedback: a Power Tool for Interactive Content-Based Image Retrieval

Relevance feedback image retrieval based on multiple features

Relevance Feedback Techniques In Interactive Content-Based Image Retrieval

A Noisy-Smoothing Relevance Feedback Method for Content-Based Medical Image Retrieval

Retrieval Of Brain Tumors By Adaptive Spatial Pooling And Fisher Vector Representation

Medical Image Retrieval with Relevance Feedback Via Pairwise Constraint Propagation.

A Novel Similarity Learning Method Via Relative Comparison for Content-Based Medical Image Retrieval

Relevance feedback: A power tool for interactive content-based image retrieval.

Content-based Image Retrieval Using Optimal Feature Combination and Relevance Feedback

Medical Image Retrieval Based on Unclean Image Bags

Content-based image retrieval system with new low-level features, new similarity metric, and novel feedback learning

Relevance Feedback Techniques and Genetic Algorithm for Image Retrieval Based on Multiple Features

Advancements in Content-Based Image Retrieval: A Comprehensive Survey of Relevance Feedback Techniques

A Comprehensive Analysis for Relevance Feedback in CBIR System.

Designing User Interfaces to Enhance Human Interpretation of Medical Content-Based Image Retrieval: Application to PET-CT Images