Abstract:Many real-world applications involve data from multiple modalities and thus exhibit the viewheterogeneity. For example, user modeling on social media might leverage both the topology of the underlying social network and the content of the users' posts; in the medical domain, multiple views could be X-ray images taken at different poses. To date, various techniques have been proposed to achieve promising results, such as canonical correlation analysis based methods, etc. In the meanwhile, it is critical for decision-makers to be able to understand the prediction results from these methods. For example, given the diagnostic result that a model provided based on the X-ray images of a patient at different poses, the doctor needs to know why the model made such a prediction. However, state-of-the-art techniques usually suffer from the inability to utilize the complementary information of each view and to explain the predictions in an interpretable manner. To address these issues, in this paper, we propose a deep coattention network for multi-view subspace learning, which aims to extract both the common information and the complementary information in an adversarial setting and provide robust interpretations behind the prediction to the end-users via the co-attention mechanism. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation by incorporating the classifier into our model. This improves the quality of latent representation and accelerates the convergence speed. Finally, we develop an efficient iterative algorithm to find the optimal encoders and discriminator, which are evaluated extensively on synthetic and real-world data sets. We also conduct a case study to demonstrate how the proposed method robustly interprets the predictions on an image data set.

Multi-view Opinion Mining with Deep Learning

A spatiotemporal multi-feature extraction framework for opinion mining

Deep Co-Attention Network for Multi-View Subspace Learning

Sentiment analysis of tweets using text and graph multi-views learning

DMTMV: A Unified Learning Framework for Deep Multi-task Multi-view Learning.

Trusted Multi-View Deep Learning with Opinion Aggregation

A Survey of Sentiment Analysis Based on Deep Learning

A Multi-model Fusion Framework Based on Deep Learning for Sentiment Classification.

Multi-view Learning Overview: Recent Progress and New Challenges

Multi-modal Multi-view Topic-opinion Mining for Social Event Analysis.

Deep Embedded Complementary and Interactive Information for Multi-View Classification

A review on multi-view learning

Interpretable deep learning methods for multiview learning

A Deep Approach For Multi-Modal User Attribute Modeling

Multi-Perspective Relevance Matching with Hierarchical ConvNets for Social Media Search

Global Inference for Aspect and Opinion Terms Co-Extraction Based on Multi-Task Neural Networks

A Hybrid Model for Opinion Mining Based on Domain Sentiment Dictionary

NeuO: Exploiting the sentimental bias between ratings and reviews with neural networks

Research of Personalized Recommendation System Based on Multi-view Deep Neural Networks

Deep Multi-View Representation Learning for Social Images

An effective hybrid model for opinion mining and sentiment analysis