Abstract:Image-Text Matching is one major task in cross-modal information processing. The main challenge is to learn the unified visual and textual representations. Previous methods that perform well on this task primarily focus on not only the alignment between region features in images and the corresponding words in sentences, but also the alignment between relations of regions and relational words. However, the lack of joint learning of regional features and global features will cause the regional features to lose contact with the global context, leading to the mismatch with those non-object words which have global meanings in some sentences. In this work, in order to alleviate this issue, it is necessary to enhance the relations between regions and the relations between regional and global concepts to obtain a more accurate visual representation so as to be better correlated to the corresponding text. Thus, a novel multi-level semantic relations enhancement approach named Dual Semantic Relations Attention Network(DSRAN) is proposed which mainly consists of two modules, separate semantic relations module and the joint semantic relations module. DSRAN performs graph attention in both modules respectively for region-level relations enhancement and regional-global relations enhancement at the same time. With these two modules, different hierarchies of semantic relations are learned simultaneously, thus promoting the image-text matching process by providing more information for the final visual representation. Quantitative experimental results have been performed on MS-COCO and Flickr30K and our method outperforms previous approaches by a large margin due to the effectiveness of the dual semantic relations learning scheme.

Image Retrieval Algorithm Based on Enhanced Relational Graph.

Image Retrieval Based on Augmented Relational Graph Representation

Incremental semi-supervised subspace learning for image retrieval.

Image retrieval based on incremental subspace learning

Learning A Semantic Space from User'S Relevance Feedback for Image Retrieval

Bayesian network based semantic image classification with attributed relational graph

Image Retrieval Based on Fuzzy Semantic Relevance Matrix

Learning an Image Manifold for Retrieval

Semi-supervised Dimensionality Reduction for Image Retrieval

An Image Retrieval Method Based On Collaborative Filtering

Enhancing Relevance Feedback in Image Retrieval Using Unlabeled Data

A Neural Network Based Self-Learning Algorithm of Image Retrieval

Semisupervised SVM Batch Mode Active Learning with Applications to Image Retrieval

A unified optimization based learning method for image retrieval

Improving Supervised Cross-modal Retrieval with Semantic Graph Embedding

A new image retrieval system supporting query by semantics and example

Recognition and Retrieval of Face Images by Semi-supervised Learning

ImageSaker : A Semantic-based Image Retrieval System Refining with Concept Model

Exploiting Unlabeled Data In Content-Based Image Retrieval

Adaptive tree similarity learning for image retrieval

Learning Dual Semantic Relations with Graph Attention for Image-Text Matching