Abstract:Automated photo tagging is essential to make massive unlabeled photos searchable by text search engines. Conventional image annotation approaches, though working reasonably well on small testbeds, are either computationally expensive or inaccurate when dealing with large-scale photo tagging. Recently, with the popularity of networking websites, we observe a massive number of user-tagged referred to as social images, that are available on the web. Unlike traditional web images often contain tags and other user-generated content, which offer a new opportunity to resolve some long-standing challenges in multimedia. In this work, we aim to address the challenge of large-scale automated photo tagging by exploring the images. We present a retrieval based approach for automated photo tagging. To tag a test image, the proposed approach first retrieves k images that share the largest visual similarity with the test image. The tags of the test image are then derived based on the tagging of the similar images. Due to the well-known semantic gap issue, a regular Euclidean distance-based retrieval method often fails to find semantically relevant images. To address the challenge of semantic gap, we propose a novel probabilistic distance metric learning scheme that (1) automatically derives constraints from the uncertain side information, and (2) efficiently learns a distance metric from the derived constraints. We apply the proposed technique to automated photo tagging tasks based on a image testbed with over 200,000 images crawled from Flickr. Encouraging results show that the proposed technique is effective and promising for automated photo tagging.

Image annotation in social networks using graph and multimodal deep learning features

Deep Attentive Multimodal Network Representation Learning for Social Media Images

The image annotation algorithm using convolutional features from intermediate layer of deep learning

A Novel Approach for Annotation-based Image Retrieval Using Deep Architecture.

The Image Annotation Method by Convolutional Features from Intermediate Layer of Deep Learning Based on Internet of Things

Image automatic annotation via multi-view deep representation

Image Tagging with Social Assistance

Annotating Images by Harnessing Worldwide User-Tagged Photos

Distance Metric Learning from Uncertain Side Information with Application to Automated Photo Tagging

Augmenting Image Processing With Social Tag Mining For Landmark Recognition

Advances in Deep Learning Approaches for Image Tagging

Transformer-Based Feature Fusion Approach for Multimodal Visual Sentiment Recognition Using Tweets in the Wild

Hybrid image analysis model for hashtag recommendation through the use of deep learning methods

Image Annotation by Latent Community Detection and Multikernel Learning.

Social image tagging using graph-based reinforcement on multi-type interrelated objects

Multi-Modal Multi-Scale Deep Learning for Large-Scale Image Annotation

Image Annotation by Object Hypotheses-oriented Deep Neural Networks

Personalized Annotation for Mobile Photos Based on User's Social Circle.

Multimodal Information Joint Learning for Geotagged Image Search.

Collaborative Filtering Motivated Automatic Photo Tagging

Visual Sentiment Analysis Using Deep Learning Models with Social Media Data