Abstract:Cross-view geo-localization refers to recognizing images of the same geographic target obtained from different platforms (such as drone-view, satellite-view, and ground-view). However, cross-view geo-localization is challenging as image capture using different platforms coupled with extreme viewpoint variations can cause significant changes to the visual image content. Existing methods mainly focus on mining the fine-grained features or the contextual information in neighboring areas, but ignore the complete information of the entire image and the association of contextual information of adjacent regions. Therefore, a multibranch joint representation learning network model based on information fusion strategies (IFSs) is proposed to solve this cross-view geo-localization problem. First, we obtained feature information from the image through global information fusion (GIF) branch and local information fusion (LIF) branch to help the network learn the discernable information in the different images. In addition, a local-guided-GIF (LGGIF) branch is introduced to make local information assist global features to enhance the learning of potential information in the images. Second, we introduced different IFSs in each branch to increase the extraction of contextual information through expanding the global receptive field, thus improving the performance of the model. Finally, a series of experiments is carried out on four prevailing benchmark datasets, namely University-1652, SUES-200, CVUAS, and CVACT datasets. The quantitative comparisons from the experiments clearly indicate that the proposed network framework has great performance. For example, compared with some state-of-the-art methods, the quantitative improvements of the R@1 and AP on the University-1652 datasets are 1.91%, 2.18%, and 1.55%, 2.99% in both tasks, respectively.

Multimodal Information Joint Learning for Geotagged Image Search.

Joint Hypergraph Learning for Tag-Based Image Retrieval

Tag-based Social Image Search with Visual-Text Joint Hypergraph Learning

Distance Metric Learning from Uncertain Side Information with Application to Automated Photo Tagging

An Efficient Approach for Geo-Multimedia Cross-Modal Retrieval

Visual-textual Joint Relevance Learning for Tag-Based Social Image Search.

Bridging the Semantic Gap Between Image Contents and Tags

Learning Neighborhood Representation from Multi-Modal Multi-Graph: Image, Text, Mobility Graph and Beyond

Integrating Multisubspace Joint Learning With Multilevel Guidance for Cross-Modal Retrieval of Remote Sensing Images

Graph-based Multimodal Semi-Supervised Image Classification

Multibranch Joint Representation Learning Based on Information Fusion Strategy for Cross-View Geo-Localization

Unsupervised Image and Text Fusion for Travel Information Enhancement

Hypergraph Spectral Hashing for image retrieval with heterogeneous social contexts

Multimodal interaction aware embedding for location-based social networks

When Location Meets Social Multimedia: A Survey on Vision-Based Recognition and Mining for Geo-Social Multimedia Analytics

Heterogeneous Metric Learning with Joint Graph Regularization for Cross-Media Retrieval.

Multimodal graph-based reranking for web image search.

MMGA: Multimodal Learning with Graph Alignment

Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking.

Multimodal collaborative graph for image recommendation

Probabilistic Hypergraph Based Hash Codes for Social Image Search