Abstract:The technology of medical data production has been rapidly changed over the past few years. Modern computer technology has created the possibility of creating multi-modal medical images. Medical data often contain multi-modal information such as visual information (image) as well as textual information. Both types of information are important for medical retrieval system (MRS). Due to the information limitation at different levels of sources, the application of information fusion becomes a real need in medical application. In this research, an information fusion framework was built to develop the multi-modality medical image retrieval system (IFM3IRS). The framework utilizes two sources of information involving text and visual-based retrieval process. The application is based on sequential order where the result from text-based process will automatically be the input in visual-based process. The main contributions of this paper are the development of a new ranking model called MedHieCon ranking model which applies semantic concepts of modality, anatomy and pathology in text-based process and also the learning approach of medical images using medical concept model in visual-based process. ImageCLEFmed 2010 data collection was used to evaluate IFM3IRS and it shows that our information fusion framework is in top list among other researchers. Although text-based retrieval system has proven to be a better performance in MRS; it is significant to determine the overall performance improvements which include the fusion of text and image.

Combining Text and Content Based Image Retrieval on Medical Resource Database

Multistage and Multi-features Medical Image Retrieval System

Research on Medical Image Retrieval System Based on DICOM Header Information and Content

A Retrieval System For 3d Multi-Phase Contrast-Enhanced Ct Images Of Focal Liver Lesions Based On Combined Bags Of Visual Words And Texture Words

Learning Based Combining Different Features for Medical Image Retrieval

Combined Density, Texture and Shape Features of Multi-phase Contrast-Enhanced CT Images for CBIR of Focal Liver Lesions: A Preliminary Study

Content-Based Retrieval of Focal Liver Lesions Using Geometrical and Textural Features of Multi-Phase CT-Scan Images

Prospective Study for Semantic Inter-Media Fusion in Content-Based Medical Image Retrieval

Multi-modal Information Retrieval for Content-based Medical Image and Video Data Mining

Improving Context-Based Medical Image Retrieval by Incorporating Semantic-Based Retrieval.

A New Two-Step Method for Medical Image Retrieval

Combining similarity measures in content-based image retrieval guided by mutual information

An Approach for Multimodal Medical Image Retrieval using Latent Dirichlet Allocation

Medical Image Retrieval with Query-Dependent Feature Fusion Based on One-Class SVM

Diagnosis system of computer-aided brain MRI using content-based image retrieval

Design and implementation of content-based medical image retrieval system on mammograms

Content-Based Image Retrieval In Picture Archiving And Communication System

Medical Image Retrieval Based on Gray Level Co-occurrence Matrix and Gradient Phase Mutual Information

Mutual Information Based Feature Selection for Medical Image Retrieval

Medical Image Retrieval System Using Multiple Features from 3d Rois

IFM3IRS: Information fusion retrieval system with knowledge-assisted text and visual features based on medical conceptual model