Abstract:As a promising field, Multi-Query Image Retrieval (MQIR) aims at searching for the semantically relevant image given multiple region-specific text queries. Existing works mainly focus on a single-level similarity between image regions and text queries, which neglect the hierarchical guidance of multi-level similarities and result in incomplete alignments. Besides, the high-level semantic correlations that intrinsically connect different region-query pairs are rarely considered. To address above limitations, we propose a novel Hierarchical Matching and Reasoning Network (HMRN) for MQIR. It disentangles MQIR into three hierarchical semantic representations, which is responsible to capture fine-grained local details, contextual global scopes, and high-level inherent correlations. HMRN consists of two modules: Scalar-based Matching (SM) module and Vector-based Reasoning (VR) module. Specifically, the SM module characterizes the multi-level alignment similarity, which consists of a fine-grained local-level similarity and a context-aware global-level similarity. Afterwards, the VR module is developed to excavate the potential semantic correlations among multiple region-query pairs, which further explores the high-level reasoning similarity. Finally, these three-level similarities are aggregated into a joint similarity space to form the ultimate similarity. Extensive experiments on the benchmark dataset demonstrate that our HMRN substantially surpasses the current state-of-the-art methods. For instance, compared with the existing best method Drill-down, the metric R@1 in the last round is improved by 23.4%. Our source codes will be released at https://github.com/LZH-053/HMRN.

Region-based Hidden Markov Models for Image Categorization and Retrieval.

Image Categorization Based on a Hierarchical Spatial Markov Model

Classification of multi-spectral remote sensing images based on Hidden Markov Models

An Image Retrieval Method Based on Region

Effective Multi-level Image Representation for Image Categorization

Algorithm of Image Classification Based on Two-Dimensional Hidden Markov Model

Region-based Image Retrieval Using Semantic Mining

Learning In Region-Based Image Retrieval

Adaptive Feature Selection and Extraction Approaches for Image Retrieval Based on Region

Probabilistic Memory Model for Visual Images Categorization.

Image Region Labeling by Exploring Contextual Information of Visual Spatial and Semantic Concepts

3D Object Classification Based on Local Keywords and Hidden Markov Model

Image Orientation Recognition Based on Hidden Markov Model

Merged Region Based Image Retrieval.

Histogram Mining, Based on Markov Chain and Its Application to Image Categorization

Support Vector Machines For Region-Based Image Retrieval

Image Segmentation Based on High-Order MRF Model with Robust Local Spatial Information

Hierarchical Markov image model and its inference algorithm

Automatic 3D Model Annotation by a Two-Dimensional Hidden Markov Model

Remotely Sensed Image Retrieval Based on Region-Level Semantic Mining

Hierarchical matching and reasoning for multi-query image retrieval