Abstract:Directly applying single-label classification methods to the multi-label learning problems substantially limits both the performance and speed due to the imbalance, dependence and high dimensionality of the given label matrix. Existing methods either ignore these three problems or reduce one with the price of aggravating another. In this paper, we propose a {0,1} label matrix compression and recovery method termed “compressed labeling (CL)” to simultaneously solve or at least reduce these three problems. CL first compresses the original label matrix to improve balance and independence by preserving the signs of its Gaussian random projections. Afterward, we directly utilize popular binary classification methods (e.g., support vector machines) for each new label. A fast recovery algorithm is developed to recover the original labels from the predicted new labels. In the recovery algorithm, a “labelset distilling method” is designed to extract distilled labelsets (DLs), i.e., the frequently appeared label subsets from the original labels via recursive clustering and subtraction. Given a distilled and an original label vector, we discover that the signs of their random projections have an explicit joint distribution that can be quickly computed from a geometric inference. Based on this observation, the original label vector is exactly determined after performing a series of Kullback-Leibler divergence based hypothesis tests on the distribution about the new labels. CL significantly improves the balance of the training samples and reduces the dependence between different labels. Moreover, it accelerates the learning process by training fewer binary classifiers for compressed labels, and makes use of label dependence via DLs based tests. Theoretically, we prove the recovery bounds of CL which verifies the effectiveness of CL for label compression and multi-label classification performance improvement brought by label correlations preserved in DLs. We show the effectiveness, efficiency and robustness of CL via 5 groups of experiments on 21 datasets from text classification, image annotation, scene classification, music categorization, genomics and web page classification.

Binary Linear Compression for Multi-label Classification.

A Bayesian Network nearest k-labels method for Multi-label classification

Compact learning for multi-label classification

Compressed labeling on distilled labelsets for multi-label learning

Towards exploiting linear regression for multi-class/multi-label classification: an empirical analysis

A Label Embedding Method via Conditional Covariance Maximization for Multi-label Classification.

Multilabel Classification Using Low-Rank Decomposition

Learning Binary Codes and Binary Weights for Efficient Classification

Learning Label Specific Features for Multi-label Classification.

Learning Compact Binary Codes from Higher-Order Tensors Via Free-Form Reshaping and Binarized Multilinear PCA.

Accurate and Efficient Large-Scale Multi-Label Learning With Reduced Feature Broad Learning System Using Label Correlation

Scalable Label Distribution Learning for Multi-Label Classification

Collaborative Multilabel Classification

Joint Learning of Multi-Label Classification and Label Correlations

Binary relevance for multi-label learning: an overview

Multi-Dimensional Classification via Decomposed Label Encoding

Joint Ranking SVM and Binary Relevance with Robust Low-Rank Learning for Multi-Label Classification

Large-Margin Learning of Compact Binary Image Encodings

ROBUST BINARY LOSS FOR MULTI-CATEGORY CLASSIFICATION WITH LABEL NOISE

Discriminative Regression With Latent Label Learning for Image Classification

Probability-based Label Enhancement for Multi-Dimensional Classification