Abstract:In biomedical articles, authors often use annotation markers such as arrows, letters, or symbols overlaid on figures and illustrations to highlight ROIs. These annotations are then referenced and correlated with concepts in the caption text or figure citations in the article text. This association creates a bridge between the visual characteristics of important regions within an image and their semantic interpretation. Identifying these assists in extracting ROIs that are likely to be highly relevant to the discussion in the article text. The aim of this work is to perform semantic search without knowing the concept keyword or the specific name of the visual pattern or appearance. We consider the problem of cross and multimodal retrieval of images from articles which contains components of text and images. Our proposed method localizes and recognizes the annotations by utilizing a combination of rule-based and statistical image processing techniques. The image regions are then annotated for classification using biomedical concepts obtained from a glossary of imaging terms. Similar automatic ROI extraction can be applied to query images, or to interactively mark an ROI. As a result, visual characteristics of the ROIs can be mapped to text concepts and then used to search image captions. In addition, the system can toggle the search process from purely perceptual to a conceptual one (crossmodal) based on utilizing user feedback or integrate both perceptual and conceptual search in a multimodal search process. The hypothesis, that such approaches would improve biomedical image retrieval, was validated through experiments on a biomedical article dataset of thoracic CT scans.

Colour Image Indexing Using SOM for Region-of-Interest Retrieval

Roi-Oriented Image Query and Indexing for Content-Based Retrieval.

AN image retrieval method based on multiple hyperspheres OC-SVM hashing

A Flower Image Retrieval Method Based on ROI Feature

Image retrieval based on incremental subspace learning

Region-of-interest based flower images retrieval

A Color Image Segmentation Method Based on SOM Network

Color Object Detection Based Image Retrieval Using ROI Segmentation with Multi-Feature Method

Interactive cross and multimodal biomedical image retrieval based on automatic region-of-interest (ROI) identification and classification

A Novel Region-Based Image Retrieval Method Using Relevance Feedback

SOM Ensemble-Based Image Segmentation

Enhanced Semantic Natural Scenery Retrieval System Through Novel Dominant Colour and Multi‐Resolution Texture Feature Learning Model

Efficient Region-Based Image Querying

Region-Based Image Retrieval Revisited

Neural maps in remote sensing image analysis

Salient Object Detection Via Google Image Retrieval

Region of Interest Extraction Via Common Salient Feature Analysis and Feedback Reinforcement Strategy for Remote Sensing Images

A Novel Image Retrieval Approach Combining Multiple Features Of Color-Connected Regions

Kansei-Oriented Image Retrieval

Relevance feedback: A power tool for interactive content-based image retrieval.

Partial-Duplicate Image Retrieval via Saliency-Guided Visual Matching