Abstract:A novel context‐aware relation enhancement and similarity reasoning model is proposed to achieve precise image‐text retrieval, which conducts both intra‐modal relation enhancement and inter‐modal similarity reasoning while considering the global‐context information. Image‐text retrieval is a fundamental yet challenging task, which aims to bridge a semantic gap between heterogeneous data to achieve precise measurements of semantic similarity. The technique of fine‐grained alignment between cross‐modal features plays a key role in various successful methods that have been proposed. Nevertheless, existing methods cannot effectively utilise intra‐modal information to enhance feature representation and lack powerful similarity reasoning to get a precise similarity score. Intending to tackle these issues, a context‐aware Relation Enhancement and Similarity Reasoning model, called RESR, is proposed, which conducts both intra‐modal relation enhancement and inter‐modal similarity reasoning while considering the global‐context information. For intra‐modal relation enhancement, a novel context‐aware graph convolutional network is introduced to enhance local feature representations by utilising relation and global‐context information. For inter‐modal similarity reasoning, local and global similarity features are exploited by the bidirectional alignment of image and text, and the similarity reasoning is implemented among multi‐granularity similarity features. Finally, refined local and global similarity features are adaptively fused to get a precise similarity score. The experimental results show that our effective model outperforms some state‐of‐the‐art approaches, achieving average improvements of 2.5% and 6.3% in R@sum on the Flickr30K and MS‐COCO dataset.

Learning Tag Relevance By Context Analysis For Social Image Retrieval

Contextual Modeling on Auxiliary Points for Robust Image Reranking

Learning A Semantic Space from User'S Relevance Feedback for Image Retrieval

Exploring Tag Relevance for Image Tag Re-Ranking

Cross-Modal Image-Tag Relevance Learning for Social Images

Topic Based Context-Aware Travel Recommendation Method Exploiting Geotagged Photos

Image Relevance Prediction Using Query-Context Bag-of-Object Retrieval Model

Visual-textual Joint Relevance Learning for Tag-Based Social Image Search.

A Semantic Context Model For Automatic Image Annotation

Learning to Tag.

Random Walks on Context-Aware Relation Graphs for Ranking Social Tags.

Learning to Rank Tags.

Context‐aware relation enhancement and similarity reasoning for image‐text retrieval

Image tag re-ranking by coupled probability transition.

Joint Hypergraph Learning for Tag-Based Image Retrieval

A Visual Analysis Approach for Community Detection of Multi-Context Mobile Social Networks

Collaborative Image Relevance Learning for Visual Re-Ranking.

Bridging the Semantic Gap Between Image Contents and Tags

Explainable Social Contextual Image Recommendation with Hierarchical Attention.

Learning based relevance feedback in image retrieval

Context-aware Image Tweet Modelling and Recommendation.