Abstract:To bridge the semantic gap between low level feature and human perception, most of the existing algorithms aim mainly at annotating images with concepts coming from only one semantic space, e.g. cognitive or affective. The naive combination of the outputs from these spaces will implicitly force the conditional independence and ignore the correlations among the spaces. In this paper, to exploit the comprehensive semantic of images, we propose a general framework for harmoniously integrating the above multiple semantics, and investigating the problem of learning to annotate images with training images labeled in two or more correlated semantic spaces, such as fascinating nighttime, or exciting cat. This kind of semantic annotation is more oriented to real world search scenario. Our proposed approach outperforms the baseline algorithms by making the following contributions. 1) Unlike previous methods that annotate images within only one semantic space, our proposed multi-semantic annotation associates each image with labels from multiple semantic spaces. 2) We develop a multi-task linear discriminative model to learn a linear mapping from features to labels. The tasks are correlated by imposing the exclusive group lasso regularization for competitive feature selection, and the graph Laplacian regularization to deal with insufficient training sample issue. 3) A Nesterov-type smoothing approximation algorithm is presented for efficient optimization of our model. Extensive experiments on NUS-WIDEEmotive dataset (56k images) with 8×81 emotive cognitive concepts and Object&Scene datasets from NUS-WIDE well validate the effectiveness of the proposed approach.

Image Annotation by Graph-Based Inference with Integrated Multiple/Single Instance Representations

Integrated graph-based semi-supervised multiple/single instance learning framework for image annotation.

Central Attention with Multi-Graphs for Image Annotation

Learning to Infer Unseen Single-/ Multi-Attribute-Object Compositions with Graph Networks.

Image Auto-annotation with Graph Learning

Multi-View Graph Embedding Learning for Image Co-Segmentation and Co-Localization

Transductive Multi-Instance Multi-Label Learning Algorithm with Application to Automatic Image Annotation

Graph-Based Semi-Supervised Learning with Multi-Label

Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition

Correlative multi-label multi-instance image annotation

Image Annotation Based on Semantic Structure and Graph Learning

Optimized Graph Learning Using Partial Tags and Multiple Features for Image and Video Annotation.

Towards Multi-Semantic Image Annotation with Graph Regularized Exclusive Group Lasso

Graph Regularized Low-Rank Feature Mapping for Multi-Label Learning with Application to Image Annotation

Hidden-concept Driven Image Decomposition Towards Semi-Supervised Multi-Label Image Annotation

Optimal Graph Learning with Partial Tags and Multiple Features for Image and Video Annotation

Unified Video Annotation Via Multigraph Learning

Multi-Modal Multi-Scale Deep Learning for Large-Scale Image Annotation

Multiple Instance Learning for Automatic Image Annotation.

A Multi-feature Fusion Method for Automatic Multi-label Image Annotation with Weighted Histogram Integral and Closure Regions Counting

Multi-graph Laplacian Feature Mapping Incorporating Tag Information for Image Annotation.