Abstract:Convolutional neural networks (CNNs) have shown great performance as general feature representations for object recognition applications. However, for multi-label images that contain multiple objects from different categories, scales and locations, global CNN features are not optimal. In this paper, we incorporate local information to enhance the feature discriminative power. In particular, we first extract object proposals from each image. With each image treated as a bag and object proposals extracted from it treated as instances, we transform the multi-label recognition problem into a multi-class multi-instance learning problem. Then, in addition to extracting the typical CNN feature representation from each proposal, we propose to make use of ground-truth bounding box annotations (strong labels) to add another level of local information by using nearest-neighbor relationships of local regions to form a multi-view pipeline. The proposed multi-view multiinstance framework utilizes both weak and strong labels effectively, and more importantly it has the generalization ability to even boost the performance of unseen categories by partial strong labels from other categories. Our framework is extensively compared with state-of-the-art handcrafted feature based methods and CNN based methods on two multi-label benchmark datasets. The experimental results validate the discriminative power and the generalization ability of the proposed framework. With strong labels, our framework is able to achieve state-of-the-art results in both datasets.

Multi-label Learning by Simultaneously Exploiting Locality Underlying the Instance Space and the Label Space

Multi-Label Learning With Local Similarity Of Samples

Multi-Label Learning by Exploiting Label Correlations Locally

Multi-Label Learning Based on Clustering Optimization

Joint Input and Output Space Learning for Multi-Label Image Classification

Multi-label learning based on neighborhood rough set label-specific features

Active Multi-label Learning with Optimal Label Subset Selection

Multi-Instance Multi-Label Learning with Application to Scene Classification

Multi-Label Low-dimensional Embedding with Missing Labels

Soft-label recover based label-specific features learning

A Multi Label Learning Method Using Affinity Propagation and Support Vector Machine

Learning Local Instance Constraint For Multi-Label Classification

MULFE: Multi-Label Learning via Label-Specific Feature Space Ensemble

Multi-label Linear Discriminant Analysis with Locality Consistency.

Joint Label-Specific Features and Correlation Information for Multi-Label Learning.

Multi-label feature selection via constraint mapping space regularization

Integrating Global and Local Feature Selection for Multi-Label Learning

Multi-label classification by exploiting label correlations

Exploit Bounding Box Annotations for Multi-Label Object Recognition

Scalable Label Distribution Learning for Multi-Label Classification

Exploiting Label Relationship in Multi-Label Learning