Abstract:Given a query photo issued by a user (q-user), the landmark retrieval is to return a set of photos with their landmarks similar to those of the query, while the existing studies on the landmark retrieval focus on exploiting geometries of landmarks for similarity matches between candidate photos and a query photo. We observe that the same landmarks provided by different users over social media community may convey different geometry information depending on the viewpoints and/or angles, and may, subsequently, yield very different results. In fact, dealing with the landmarks with low quality shapes caused by the photography of q-users is often nontrivial and has seldom been studied. In this paper, we propose a novel framework, namely, multi-query expansions, to retrieve semantically robust landmarks by two steps. First, we identify the top- k photos regarding the latent topics of a query landmark to construct multi-query set so as to remedy its possible low quality shape. For this purpose, we significantly extend the techniques of Latent Dirichlet Allocation. Then, motivated by the typical collaborative filtering methods, we propose to learn a collaborative deep networks-based semantically, nonlinear, and high-level features over the latent factor for landmark photo as the training set, which is formed by matrix factorization over collaborative user-photo matrix regarding the multi-query set. The learned deep network is further applied to generate the features for all the other photos, meanwhile resulting into a compact multi-query set within such space. Then, the final ranking scores are calculated over the high-level feature space between the multi-query set and all other photos, which are ranked to serve as the final ranking list of landmark retrieval. Extensive experiments are conducted on real-world social media data with both landmark photos together with their user information to show the superior performance over the existing methods, especially our recently proposed multi-query based mid-level pattern representation method [1].

Query by Strings and Return Ranking Word Regions with Only One Look

Keyword Spotting Simplified: A Segmentation-Free Approach using Character Counting and CTC re-scoring

TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision

SpottingNet: Learning the Similarity of Word Images with Convolutional Neural Network for Word Spotting in Handwritten Historical Documents

A Multi-oriented Chinese Keyword Spotter Guided by Text Line Detection

Bag-Of-Words Based Deep Neural Network For Image Retrieval

Towards End-to-End Text Spotting in Natural Scenes

Rethinking Two-Stage Referring Expression Comprehension: A Novel Grounding and Segmentation Method Modulated by Point

Word Image Representation Based On Visual Embeddings And Spatial Constraints For Keyword Spotting On Historical Documents

Out-domain Chinese New Word Detection with Statistics-Based Character Embedding

Page-Level Handwritten Word Spotting via Discriminative Feature Learning.

Key-Word-Aware Network for Referring Expression Image Segmentation

Unsupervised Query Segmentation Using Monolingual Word Alignment Method.

A Query-Based Network for Rural Homestead Extraction from VHR Remote Sensing Images

Deep Learning for Chinese Word Segmentation and POS Tagging.

HENet: Hyperbolic-Based Encoder-Decoder Network for Word Spotting in Historical Mongolian Documents

A Hybrid Representation of Word Images for Keyword Spotting.

Effective Multi-Query Expansions: Collaborative Deep Networks for Robust Landmark Retrieval

Word Image Representation Based on Sequence to Sequence Model with Attention Mechanism for Out-of-Vocabulary Keyword Spotting.

Realization of Chinese Word Segmentation Based on Deep Learning Method