Abstract:In this work, we investigate how to reassign the fully annotated labels at image level to those contextually derived semantic regions, namely Label-to-Region (L2R), in a collective manner. Given a set of input images with label annotations, the basic idea of our approach to L2R is to first discover the patch correspondence across images, and then propagate the common labels shared in image pairs to these correlated patches. Specially, our approach consists of following aspects. First, each of the input images is encoded as a Bag-of-Hierarchical-Patch (BOP) for capturing the rich cues at variant scales, and the individual patches are expressed by patch-level feature descriptors. Second, we present a sparse representation formulation for discovering how well an image or a semantic region can be robustly reconstructed by all the other image patches from the input image set. The underlying philosophy of our formulation is that an image region can be sparsely reconstructed with the image patches belonging to the other images with common labels, while the robustness in label propagation across images requires that these selected patches come from very few images. This preference of being sparse at both patch and image level is named bi-layer sparsity prior . Meanwhile, we enforce the preference of choosing larger-size patches in reconstruction, referred to as continuity-biased prior in this work, which may further enhance the reliability of L2R assignment. Finally, we harness the reconstruction coefficients to propagate the image labels to the matched patches, and fuse the propagation results over all patches to finalize the L2R task. As a by-product, the proposed continuity-biased bi-layer sparse representation formulation can be naturally applied to perform image annotation on new testing images. Extensive experiments on three public image datasets clearly demonstrate the effectiveness of our proposed framework in both L2R assignment and image annotation.

Nonparametric Label-to-Region by search

Label-to-region with Continuity-Biased Bi-Layer Sparsity Priors

Learning with Both Unlabeled Data and Query Logs for Image Search.

Label to Region by BiLayer Sparsity Priors

Efficient Region-Aware Large Graph Construction Towards Scalable Multi-Label Propagation

Image region label refinement using spatial position relation graph.

AN EM-BASED UNSUPERVISED REGIONS ANNOTATION ALGORITHM FOR MULTI-LABELS OF IMAGE

Hidden-Concept Driven Multilabel Image Annotation and Label Ranking

Image Region Labeling by Exploring Contextual Information of Visual Spatial and Semantic Concepts

Local Image Tagging Via Graph Regularized Joint Group Sparsity

Correlated Probabilistic Label Propagation for Region-Based Image Retrieval

Multilabel Neighborhood Propagation for Region-Based Image Retrieval

Automatic tag-to-region assignment via multiple instance learning

Region-based supervised annotation for semantic image retrieval

Using Label Propagation to Get Confidence Map for Segmentation

Visual and textual fusion for semantically supervised region-based retrieval

Label Propagation Through Linear Neighborhoods

Social tag alignment with image regions by sparse reconstructions.

Correlative multi-label multi-instance image annotation

Semi-Supervised Classification Using Linear Neighborhood Propagation

Synergy-Guided Regional Supervision of Pseudo Labels for Semi-Supervised Medical Image Segmentation