Abstract:As we all know, multi-view data is more expressive than single-view data and multi-label annotation enjoys richer supervision information than single-label, which makes multi-view multi-label learning widely applicable for various pattern recognition tasks. In this complex representation learning problem, three main challenges can be characterized as follows: i) How to learn consistent representations of samples across all views? ii) How to exploit and utilize category correlations of multi-label to guide inference? iii) How to avoid the negative impact resulting from the incompleteness of views or labels? To cope with these problems, we propose a general multi-view multi-label learning framework named label-guided masked view- and category-aware transformers in this paper. First, we design two transformer-style based modules for cross-view features aggregation and multi-label classification, respectively. The former aggregates information from different views in the process of extracting view-specific features, and the latter learns subcategory embedding to improve classification performance. Second, considering the imbalance of expressive power among views, an adaptively weighted view fusion module is proposed to obtain view-consistent embedding features. Third, we impose a label manifold constraint in sample-level representation learning to maximize the utilization of supervised information. Last but not least, all the modules are designed under the premise of incomplete views and labels, which makes our method adaptable to arbitrary multi-view and multi-label data. Extensive experiments on five datasets confirm that our method has clear advantages over other state-of-the-art methods.

Unlabeled Data and Multiple Views

Learning with Both Unlabeled Data and Query Logs for Image Search.

Learning with Unlabeled Data and Its Application to Image Retrieval

Multiview Latent Space Learning with Progressively Fine-tuned Deep Features for Unsupervised Domain Adaptation

Deep Double Incomplete Multi-View Multi-Label Learning With Incomplete Labels and Missing Views

Multi-View Learning with Incomplete Views

A Survey on Multi-view Learning

Incomplete multi-view learning: Review, analysis, and prospects

Multiple View Semi-Supervised Dimensionality Reduction

Deep Incomplete Multi-View Learning Network with Insufficient Label Information

Multi-Instance Learning from Supervised View

Unlabeled data selection for active learning in image classification

Multi-View Embedding Learning for Incompletely Labeled Data.

Exploiting Unlabeled Data via Partial Label Assignment for Multi-Class Semi-Supervised Learning

Incomplete Multi-View Multi-Label Learning via Label-Guided Masked View- and Category-Aware Transformers

Incomplete Multi-View Weak-Label Learning with Noisy Features and Imbalanced Labels

Semi-supervised learning with very few labeled training examples

Unsupervised Feature Learning in Remote Sensing

Exploiting Label Relationship in Multi-Label Learning

Few-Shot Partial Multi-View Learning

Global and local multi-view multi-label learning with incomplete views and labels