Abstract:Many real-world applications involve data from multiple modalities and thus exhibit the viewheterogeneity. For example, user modeling on social media might leverage both the topology of the underlying social network and the content of the users' posts; in the medical domain, multiple views could be X-ray images taken at different poses. To date, various techniques have been proposed to achieve promising results, such as canonical correlation analysis based methods, etc. In the meanwhile, it is critical for decision-makers to be able to understand the prediction results from these methods. For example, given the diagnostic result that a model provided based on the X-ray images of a patient at different poses, the doctor needs to know why the model made such a prediction. However, state-of-the-art techniques usually suffer from the inability to utilize the complementary information of each view and to explain the predictions in an interpretable manner. To address these issues, in this paper, we propose a deep coattention network for multi-view subspace learning, which aims to extract both the common information and the complementary information in an adversarial setting and provide robust interpretations behind the prediction to the end-users via the co-attention mechanism. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation by incorporating the classifier into our model. This improves the quality of latent representation and accelerates the convergence speed. Finally, we develop an efficient iterative algorithm to find the optimal encoders and discriminator, which are evaluated extensively on synthetic and real-world data sets. We also conduct a case study to demonstrate how the proposed method robustly interprets the predictions on an image data set.

Attention-Induced Embedding Imputation for Incomplete Multi-View Partial Multi-Label Classification

Deep Double Incomplete Multi-View Multi-Label Learning With Incomplete Labels and Missing Views

Dual Enhancement for Multi-Label Learning with Missing Labels

Task-Augmented Cross-View Imputation Network for Partial Multi-View Incomplete Multi-Label Classification

Incomplete Multi-View Partial Multi-Label Learning.

Partial Multi-View Multi-Label Classification Via Semantic Invariance Learning and Prototype Modeling

Multi-View Embedding Learning for Incompletely Labeled Data.

A Two-Stage Information Extraction Network for Incomplete Multi-View Multi-Label Classification

Incomplete Multi-View Weak-Label Learning with Noisy Features and Imbalanced Labels

Incomplete Multi-view Multi-label Classification via a Dual-level Contrastive Learning Framework

Multi-scale locality preserving projection for partial multi-view incomplete multi-label learning

Deep dual incomplete multi-view multi-label classification via label semantic-guided contrastive learning

Improved Multi-view Multi-label Learning with Incomplete Views and Labels

Deep Incomplete Multi-View Network Semi-Supervised Multi-Label Learning with Unbiased Loss

Deep Incomplete Multi-View Learning Network with Insufficient Label Information

Masked Two-channel Decoupling Framework for Incomplete Multi-view Weak Multi-label Learning

Latent Semantic Aware Multi-View Multi-Label Classification

Deep Co-Attention Network for Multi-View Subspace Learning

Incomplete Multi-View Partial Multi-Label Classification Via Deep Semantic Structure Preservation

Reliable Representations Learning for Incomplete Multi-View Partial Multi-Label Classification

Deep Embedded Complementary and Interactive Information for Multi-View Classification