Abstract:BACKGROUND: The majority of current medical CBIR systems perform retrieval based only on "imaging signatures" generated by extracting pixel-level quantitative features, and only rarely has a feedback mechanism been incorporated to improve retrieval performance. In addition, current medical CBIR approaches do not routinely incorporate semantic terms that model the user's high-level expectations, and this can limit CBIR performance.METHOD: We propose a retrieval framework that exploits a hybrid feature space (HFS) that is built by integrating low-level image features and high-level semantic terms, through rounds of relevance feedback (RF) and performs similarity-based retrieval to support semi-automatic image interpretation. The novelty of the proposed system is that it can impute the semantic features of the query image by reformulating the query vector representation in the HFS via user feedback. We implemented our framework as a prototype that performs the retrieval over a database of 811 radiographic images that contains 69 unique types of bone tumors.RESULTS: We evaluated the system performance by conducting independent reading sessions with two subspecialist musculoskeletal radiologists. For the test set, the proposed retrieval system at fourth RF iteration of the sessions conducted with both the radiologists achieved mean average precision (MAP) value ∼0.90 where the initial MAP with baseline CBIR was 0.20. In addition, we also achieved high prediction accuracy (>0.8) for the majority of the semantic features automatically predicted by the system.CONCLUSION: Our proposed framework addresses some limitations of existing CBIR systems by incorporating user feedback and simultaneously predicting the semantic features of the query image. This obviates the need for the user to provide those terms and makes CBIR search more efficient for inexperience users/trainees. Encouraging results achieved in the current study highlight possible new directions in radiological image interpretation employing semantic CBIR combined with relevance feedback of visual similarity.

A comprehensive review of content-based image retrieval systems using deep learning and hand-crafted features in medical imaging: Research challenges and future directions

Multistage and Multi-features Medical Image Retrieval System

Using Deep Learning for Content-Based Medical Image Retrieval

Deep Learning for Content-Based Image Retrieval: A Comprehensive Study

An effective image retrieval system using machine learning and fuzzy c- means clustering approach

Content-based medical image retrieval using deep learning-based features and hybrid meta-heuristic optimization

A Survey on Relevance Feedback Techniques in Content-based Medical Image Retrieval

Application of Content-Based Image Retrieval in Medical Image Acquisition

Improved Content Based Image Retrieval Process Based on Deep Convolutional Neural Network and Salp Swarm Algorithm

Advancements in Content-Based Image Retrieval: A Comprehensive Survey of Relevance Feedback Techniques

Content-Based Image Retrieval for Multi-Class Volumetric Radiology Images: A Benchmark Study

Deep Image Retrieval: A Survey

A New Approach for Effective Retrieval of Medical Images: A Step towards Computer-Assisted Diagnosis

Medical Image Retrieval using Deep Convolutional Neural Network

A comprehensive guide to content-based image retrieval algorithms with visualsift ensembling

A deep neural network model for content-based medical image retrieval with multi-view classification

Content-Based Retrieval Of 3d Medical Images

A Semantic Image Retrieval Approach Between Visual Features and Medical Concepts

Relevance feedback for enhancing content based image retrieval and automatic prediction of semantic image features: Application to bone tumor radiographs

Medical Image Retrieval Using Empirical Mode Decomposition with Deep Convolutional Neural Network

Different Types of CBIR Applications : A Survey