Abstract:Nowadays the locations of social images play an important role in geographic knowledge discovery. However, most social images still lack the location information, driving location estimation for social images to have recently become an active research topic. With the rapid growth of social images, new challenges have been posed: 1) data quality of social images is an issue because they are often associated with noises and error-prone user-generated content, such as junk comments and misspelled words; and 2) data sparsity exists in social images despite the large volume, since most of them are unevenly distributed around the world and their contextual information is often missing or incomplete. In this paper, we propose a spatial-aware multimodal location estimation (SMLE) framework to tackle the above challenges. Specifically, a spatial-aware language model (SLM) is proposed to detect the high quality location-indicative tags from large datasets. We also design a spatial-aware topic model, namely spatial-aware regularized latent semantic indexing (SRLSI), to discover geographic topics and alleviate the data sparseness problem existing in language modeling. Taking multi-modalities of social images into consideration, we employ the learning to rank approach to fuse multiple evidences derived from textual features represented by SLM and SRLSI, and visual features represented by bag-of-visual-words (BoVW). Importantly, an ad hoc method is introduced to construct the training dataset with spatial-aware relevance labels for learning to rank training. Finally, given a query image, its location is estimated as the location of its most relevant image returned from the learning to rank model. The proposed framework is evaluated on a public benchmark provided by MediaEval 2013 Placing Task, which contains more than 8.5 million images crawled from Flickr. Extensive experiments on this dataset demonstrate the superior performance of the proposed methods over the state-of-the-art approaches.

Tag Localization with Spatial Correlations and Joint Group Sparsity

Joint CRF and Locality-Consistent Dictionary Learning for Semantic Segmentation.

Local Image Tagging Via Graph Regularized Joint Group Sparsity

Social tag alignment with image regions by sparse reconstructions.

Locality-aware Group Sparse Coding on Grassmann Manifolds for Image Set Classification

Spatial-Aware Multimodal Location Estimation For Social Images

Label localization with weakly spatial constrained graph propagation.

Graph-guided sparse reconstruction for region tagging

SSA-Net: Spatial Scale Attention Network for Image-Based Geo-Localization

Spatial Constraint for Image Location Estimation.

Image Taken Place Estimation via Geometric Constrained Spatial Layer Matching.

Local and Global Regularized Sparse Coding for Data Representation

Reconciliation of Statistical and Spatial Sparsity For Robust Image and Image-Set Classification

Adaptive Gaussian Regularization Constrained Sparse Subspace Clustering for Image Segmentation.

A Locality Sensitive Low-Rank Model for Image Tag Completion

Spatially Constrained Sparse Coding Scheme for Natural Scene Categorization

Image Location Estimation by Salient Region Matching

Exploiting Spatial Context Constraints for Automatic Image Region Annotation.

Noisy Tag Alignment with Image Regions.

Image annotation refinement via 2P-KNN based group sparse reconstruction

Nonconvex submodule clustering via joint sliced sparse gradient and cluster-aware approach