Abstract:A novel context‐aware relation enhancement and similarity reasoning model is proposed to achieve precise image‐text retrieval, which conducts both intra‐modal relation enhancement and inter‐modal similarity reasoning while considering the global‐context information. Image‐text retrieval is a fundamental yet challenging task, which aims to bridge a semantic gap between heterogeneous data to achieve precise measurements of semantic similarity. The technique of fine‐grained alignment between cross‐modal features plays a key role in various successful methods that have been proposed. Nevertheless, existing methods cannot effectively utilise intra‐modal information to enhance feature representation and lack powerful similarity reasoning to get a precise similarity score. Intending to tackle these issues, a context‐aware Relation Enhancement and Similarity Reasoning model, called RESR, is proposed, which conducts both intra‐modal relation enhancement and inter‐modal similarity reasoning while considering the global‐context information. For intra‐modal relation enhancement, a novel context‐aware graph convolutional network is introduced to enhance local feature representations by utilising relation and global‐context information. For inter‐modal similarity reasoning, local and global similarity features are exploited by the bidirectional alignment of image and text, and the similarity reasoning is implemented among multi‐granularity similarity features. Finally, refined local and global similarity features are adaptively fused to get a precise similarity score. The experimental results show that our effective model outperforms some state‐of‐the‐art approaches, achieving average improvements of 2.5% and 6.3% in R@sum on the Flickr30K and MS‐COCO dataset.

Combining Content and Context Similarities for Image Retrieval

SIFT-Based Image Retrieval Combining the Distance Measure of Global Image and Sub-Image

Contextual Modeling on Auxiliary Points for Robust Image Reranking

Combining similarity measures in content-based image retrieval guided by mutual information

Contextual Visual Similarity

Improving Context-Based Medical Image Retrieval by Incorporating Semantic-Based Retrieval.

Context‐aware relation enhancement and similarity reasoning for image‐text retrieval

Object-Based Image Retrieval Using Spatial Context

Feature-Based Similarity Retrieval in Content-Based Image Retrieval

Deep Learning for Content-Based Image Retrieval: A Comprehensive Study

A parameterized evaluation model of similarity on multi-target retrieval for remote sensing image

Contextual Hashing for Large-Scale Image Search.

A Region-based Image Similarity Computation Approach

Image Similarity using An Ensemble of Context-Sensitive Models

Content-based Image Retrieval Using Optimal Feature Combination and Relevance Feedback

Improving Performance of Colour-Histogram-Based CBIR Using Bin Matching for Similarity Measure

Context-Aware Attention Network for Image-Text Retrieval

Rank-Aware Graph Fusion With Contextual Dissimilarity Measurement For Image Retrieval

Enhancing Remote Sensing Image Retrieval: A Hierarchical Approach Integrating Visual and Semantic Similarities

Towards Context-Aware Evaluation for Image Search

Exploring Statistical Correlations for Image Retrieval