Learning Domain Representation for Multi-Domain Sentiment Classification.

Qi Liu,Yue Zhang,Jiangming Liu

DOI: https://doi.org/10.18653/v1/n18-1050

2018-01-01

Abstract:Qi Liu, Yue Zhang, Jiangming Liu. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

What problem does this paper attempt to address?

Domain Attention Model for Multi-Domain Sentiment Classification.

Zhigang Yuan,Sixing Wu,Fangzhao Wu,Junxin Liu,Yongfeng Huang

DOI: https://doi.org/10.1016/j.knosys.2018.05.004

IF: 8.139

2018-01-01

Knowledge-Based Systems

Abstract:Sentiment classification is widely known as a domain-dependent problem. In order to learn an accurate domain-specific sentiment classifier, a large number of labeled samples are needed, which are expensive and time-consuming to annotate. Multi-domain sentiment analysis based on multi-task learning can leverage labeled samples in each single domain, which can alleviate the need for large amount of labeled data in all domains. In this paper, we propose a domain attention model for multi-domain sentiment analysis. In our approach, the domain representation is used as attention to select the most domain-related features in each domain. The domain representation is obtained through an auxiliary domain classification task, which works as domain regularizer. In this way, both shared and domain-specific features for sentiment classification are extracted simultaneously. In contrast with existing multi-domain sentiment classification methods, our approach can extract the most discriminative features from a shared hidden layer in a more compact way. Experimental results on two multi-domain sentiment datasets validate the effectiveness of our approach.
Multi-Domain Sentiment Classification Based on Domain-Aware Embedding and Attention

Yitao Cai,Xiaojun Wan

DOI: https://doi.org/10.24963/ijcai.2019/681

2019-01-01

Abstract:Sentiment classification is a fundamental task in NLP. However, as revealed by many researches, sentiment classification models are highly domain-dependent. It is worth investigating to leverage data from different domains to improve the classification performance in each domain. In this work, we propose a novel completely-shared multi-domain neural sentiment classification model to learn domain-aware word embeddings and make use of domain-aware attention mechanism. Our model first utilizes BiLSTM for domain classification and extracts domain-specific features for words, which are then combined with general word embeddings to form domain-aware word embeddings. Domain-aware word embeddings are fed into another BiLSTM to extract sentence features. The domain-aware attention mechanism is used for selecting significant features, by using the domain-aware sentence representation as the query vector. Evaluation results on public datasets with 16 different domains demonstrate the efficacy of our proposed model. Further experiments show the generalization ability and the transferability of our model.
Domain adaptive learning for multi realm sentiment classification on big data

Maha Ijaz,Naveed Anwar,Mejdl Safran,Sultan Alfarhood,Tariq Sadad,Imran

DOI: https://doi.org/10.1371/journal.pone.0297028

IF: 3.7

2024-04-01

PLoS ONE

Abstract:Machine learning techniques that rely on textual features or sentiment lexicons can lead to erroneous sentiment analysis. These techniques are especially vulnerable to domain-related difficulties, especially when dealing in Big data. In addition, labeling is time-consuming and supervised machine learning algorithms often lack labeled data. Transfer learning can help save time and obtain high performance with fewer datasets in this field. To cope this, we used a transfer learning-based Multi-Domain Sentiment Classification (MDSC) technique. We are able to identify the sentiment polarity of text in a target domain that is unlabeled by looking at reviews in a labelled source domain. This research aims to evaluate the impact of domain adaptation and measure the extent to which transfer learning enhances sentiment analysis outcomes. We employed transfer learning models BERT, RoBERTa, ELECTRA, and ULMFiT to improve the performance in sentiment analysis. We analyzed sentiment through various transformer models and compared the performance of LSTM and CNN. The experiments are carried on five publicly available sentiment analysis datasets, namely Hotel Reviews (HR), Movie Reviews (MR), Sentiment140 Tweets (ST), Citation Sentiment Corpus (CSC), and Bioinformatics Citation Corpus (BCC), to adapt multi-target domains. The performance of numerous models employing transfer learning from diverse datasets demonstrating how various factors influence the outputs.

multidisciplinary sciences
Collaborative Multi-domain Sentiment Classification

Fangzhao Wu,Yongfeng Huang

DOI: https://doi.org/10.1109/ICDM.2015.68

2015-01-01

Abstract:Sentiment classification is a hot research topic in both industrial and academic fields. The mainstream sentiment classification methods are based on machine learning and treat sentiment classification as a text classification problem. However, sentiment classification is widely recognized as a highly domain-dependent task. The sentiment classifier trained in one domain may not perform well in another domain. A simple solution to this problem is training a domain-specific sentiment classifier for each domain. However, it is difficult to label enough data for every domain since they are in a large quantity. In addition, this method omits the sentiment information in other domains. In this paper, we propose to train sentiment classifiers for multiple domains in a collaborative way based on multi-task learning. Specifically, we decompose the sentiment classifier in each domain into two components, a general one and a domain-specific one. The general sentiment classifier can capture the global sentiment information and is trained across various domains to obtain better generalization ability. The domain-specific sentiment classifier is trained using the labeled data in one domain to capture the domain-specific sentiment information. In addition, we explore two kinds of relations between domains, one based on textual content and the other one based on sentiment word distribution. We build a domain similarity graph using domain relations and encode it into our approach as regularization over the domain-specific sentiment classifiers. Besides, we incorporate the sentiment knowledge extracted from sentiment lexicons to help train the general sentiment classifier more accurately. Moreover, we introduce an accelerated optimization algorithm to train the sentiment classifiers efficiently. Experimental results on two benchmark sentiment datasets show that our method can outperform baseline methods significantly and consistently.
Incorporating Domain and Sentiment Supervision in Representation Learning for Domain Adaptation

Biao Liu,Minlie Huang,Jiashen Sun,Xuan Zhu

2015-01-01

Abstract:Domain adaptation aims at learning robust classifiers across domains using labeled data from a source domain. Representation learning methods, which project the original features to a new feature space, have been proved to be quite effective for this task. However, these unsupervised methods neglect the domain information of the input and are not specialized for the classification task. In this work, we address two key factors to guide the representation learning process for domain adaptation of sentiment classification -- one is domain supervision, enforcing the learned representation to better predict the domain of an input, and the other is sentiment supervision which utilizes the source domain sentiment labels to learn sentiment-favorable representations. Experimental results show that these two factors significantly improve the proposed models as expected.
Supervised Domain Adaptation for Sentiment Regression.

Jian Xu,Hao Yin,Shoushan Li,Guodong Zhou

DOI: https://doi.org/10.1007/978-981-10-6805-8_16

2017-01-01

Abstract:Sentiment regression is a task of summarizing the overall sentiment of a review with a real-valued score. However, the regression model trained in one domain probably performs poorly in a different domain due to the distribution variety. Different from existing studies, domain adaptation in sentiment regression is more challenging because the rating range in one domain might be different from that in the other domain. In this study, we propose a novel approach to domain adaptation for sentiment regression. Specifically, our approach employs an auxiliary Long Short-Term Memory (LSTM) layer to learn the auxiliary representation from the source domain, and simultaneously join the auxiliary representation into the main LSTM layer for the target domain regression setting. In the learning process, the LSTM regression models for the source and target domains are jointly learned. Empirical studies demonstrate that our joint learning approach performs significantly better than several strong baselines.
Domain-Aware Attention FC yDomain Feature Extractor and Classifier Sentiment Feature Extractor and ClassifierDomain-Aware Query VectorBiLSTM BiLSTM Domain-Aware Word Embedding

Yitao Cai,Xiaojun Wan

2019-01-01

Abstract:Sentiment classification is a fundamental task in NLP. However, as revealed by many researches, sentiment classification models are highly domaindependent. It is worth investigating to leverage data from different domains to improve the classification performance in each domain. In this work, we propose a novel completely-shared multidomain neural sentiment classification model to learn domain-aware word embeddings and make use of domain-aware attention mechanism. Our model first utilizes BiLSTM for domain classification and extracts domain-specific features for words, which are then combined with general word embeddings to form domain-aware word embeddings. Domain-aware word embeddings are fed into another BiLSTM to extract sentence features. The domain-aware attention mechanism is used for selecting significant features, by using the domainaware sentence representation as the query vector. Evaluation results on public datasets with 16 different domains demonstrate the efficacy of our proposed model. Further experiments show the generalization ability and the transferability of our model.
Cross-Lingual Sentiment Classification with Bilingual Document Representation Learning

Xinjie Zhou,Xianjun Wan,Jianguo Xiao

DOI: https://doi.org/10.18653/v1/p16-1133

2016-01-01

Abstract:Cross-lingual sentiment classification aims to adapt the sentiment resource in a resource-rich language to a resource-poor language. In this study, we propose a representation learning approach which simultaneously learns vector representations for the texts in both the source and the target languages. Different from previous research which only gets bilingual word embedding, our Bilingual Document Representation Learning model BiDRL directly learns document representations. Both semantic and sentiment correlations are utilized to map the bilingual texts into the same embedding space. The experiments are based on the multilingual multi-domain Amazon review dataset. We use English as the source language and use Japanese, German and French as the target languages. The experimental results show that BiDRL outperforms the state-of-the-art methods for all the target languages.
Weighed Domain-Invariant Representation Learning for Cross-domain Sentiment Analysis

Minlong Peng,Qi Zhang,Xuanjing Huang

DOI: https://doi.org/10.48550/arXiv.1909.08167

2019-09-18

Abstract:Cross-domain sentiment analysis is currently a hot topic in the research and engineering areas. One of the most popular frameworks in this field is the domain-invariant representation learning (DIRL) paradigm, which aims to learn a distribution-invariant feature representation across domains. However, in this work, we find out that applying DIRL may harm domain adaptation when the label distribution $\rm{P}(\rm{Y})$ changes across domains. To address this problem, we propose a modification to DIRL, obtaining a novel weighted domain-invariant representation learning (WDIRL) framework. We show that it is easy to transfer existing SOTA DIRL models to WDIRL. Empirical studies on extensive cross-domain sentiment analysis tasks verified our statements and showed the effectiveness of our proposed solution.

Machine Learning,Computation and Language
Collaboratively Training Sentiment Classifiers for Multiple Domains

Fangzhao Wu,Zhigang Yuan,Yongfeng Huang

DOI: https://doi.org/10.1109/tkde.2017.2669975

IF: 9.235

2017-07-01

IEEE Transactions on Knowledge and Data Engineering

Abstract:We propose a collaborative multi-domain sentiment classification approach to train sentiment classifiers for multiple domains simultaneously. In our approach, the sentiment information in different domains is shared to train more accurate and robust sentiment classifiers for each domain when labeled data is scarce. Specifically, we decompose the sentiment classifier of each domain into two components, a global one and a domain-specific one. The global model can capture the general sentiment knowledge and is shared by various domains. The domain-specific model can capture the specific sentiment expressions in each domain. In addition, we extract domain-specific sentiment knowledge from both labeled and unlabeled samples in each domain and use it to enhance the learning of domain-specific sentiment classifiers. Besides, we incorporate the similarities between domains into our approach as regularization over the domain-specific sentiment classifiers to encourage the sharing of sentiment information between similar domains. Two kinds of domain similarity measures are explored, one based on textual content and the other one based on sentiment expressions. Moreover, we introduce two efficient algorithms to solve the model of our approach. Experimental results on benchmark datasets show that our approach can effectively improve the performance of multi-domain sentiment classification and significantly outperform baseline methods.

computer science, information systems, artificial intelligence,engineering, electrical & electronic
Multi-domain active learning for text classification.

Lianghao Li,Xiaoming Jin,Sinno Jialin Pan,Jian-Tao Sun

DOI: https://doi.org/10.1145/2339530.2339701

2012-01-01

Abstract:ABSTRACTActive learning has been proven to be effective in reducing labeling efforts for supervised learning. However, existing active learning work has mainly focused on training models for a single domain. In practical applications, it is common to simultaneously train classifiers for multiple domains. For example, some merchant web sites (like Amazon.com) may need a set of classifiers to predict the sentiment polarity of product reviews collected from various domains (e.g., electronics, books, shoes). Though different domains have their own unique features, they may share some common latent features. If we apply active learning on each domain separately, some data instances selected from different domains may contain duplicate knowledge due to the common features. Therefore, how to choose the data from multiple domains to label is crucial to further reducing the human labeling efforts in multi-domain learning. In this paper, we propose a novel multi-domain active learning framework to jointly select data instances from all domains with duplicate information considered. In our solution, a shared subspace is first learned to represent common latent features of different domains. By considering the common and the domain-specific features together, the model loss reduction induced by each data instance can be decomposed into a common part and a domain-specific part. In this way, the duplicate information across domains can be encoded into the common part of model loss reduction and taken into account when querying. We compare our method with the state-of-the-art active learning approaches on several text classification tasks: sentiment classification, newsgroup classification and email spam filtering. The experiment results show that our method reduces the human labeling efforts by 33.2%, 42.9% and 68.7% on the three tasks, respectively.
Learning Sentence Representations over Tree Structures for Target-Dependent Classification.

Junwen Duan,Xiao Ding,Ting Liu

DOI: https://doi.org/10.18653/v1/n18-1051

2018-01-01

Abstract:Junwen Duan, Xiao Ding, Ting Liu. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.
Improving Domain-Adapted Sentiment Classification by Deep Adversarial Mutual Learning

Qianming Xue,Wei Zhang,Hongyuan Zha

DOI: https://doi.org/10.1609/aaai.v34i05.6477

2020-01-01

Proceedings of the AAAI Conference on Artificial Intelligence

Abstract:Domain-adapted sentiment classification refers to training on a labeled source domain to well infer document-level sentiment on an unlabeled target domain. Most existing relevant models involve a feature extractor and a sentiment classifier, where the feature extractor works towards learning domain-invariant features from both domains, and the sentiment classifier is trained only on the source domain to guide the feature extractor. As such, they lack a mechanism to use sentiment polarity lying in the target domain. To improve domain-adapted sentiment classification by learning sentiment from the target domain as well, we devise a novel deep adversarial mutual learning approach involving two groups of feature extractors, domain discriminators, sentiment classifiers, and label probers. The domain discriminators enable the feature extractors to obtain domain-invariant features. Meanwhile, the label prober in each group explores document sentiment polarity of the target domain through the sentiment prediction generated by the classifier in the peer group, and guides the learning of the feature extractor in its own group. The proposed approach achieves the mutual learning of the two groups in an end-to-end manner. Experiments on multiple public datasets indicate our method obtains the state-of-the-art performance, validating the effectiveness of mutual learning through label probers.
Emoji-Powered Representation Learning for Cross-Lingual Sentiment Classification

Zhenpeng Chen,Sheng Shen,Ziniu Hu,Xuan Lu,Qiaozhu Mei,Xuanzhe Liu

DOI: https://doi.org/10.48550/arXiv.1806.02557

2019-03-25

Abstract:Sentiment classification typically relies on a large amount of labeled data. In practice, the availability of labels is highly imbalanced among different languages, e.g., more English texts are labeled than texts in any other languages, which creates a considerable inequality in the quality of related information services received by users speaking different languages. To tackle this problem, cross-lingual sentiment classification approaches aim to transfer knowledge learned from one language that has abundant labeled examples (i.e., the source language, usually English) to another language with fewer labels (i.e., the target language). The source and the target languages are usually bridged through off-the-shelf machine translation tools. Through such a channel, cross-language sentiment patterns can be successfully learned from English and transferred into the target languages. This approach, however, often fails to capture sentiment knowledge specific to the target language, and thus compromises the accuracy of the downstream classification task. In this paper, we employ emojis, which are widely available in many languages, as a new channel to learn both the cross-language and the language-specific sentiment patterns. We propose a novel representation learning method that uses emoji prediction as an instrument to learn respective sentiment-aware representations for each language. The learned representations are then integrated to facilitate cross-lingual sentiment classification. The proposed method demonstrates state-of-the-art performance on benchmark datasets, which is sustained even when sentiment labels are scarce.

Information Retrieval
Multi-Instance Multi-Label Learning Networks for Aspect-Category Sentiment Analysis

Yuncong Li,Cunxiang Yin,Sheng-hua Zhong,Xu Pan

DOI: https://doi.org/10.48550/arXiv.2010.02656

2020-10-06

Computation and Language

Abstract:Aspect-category sentiment analysis (ACSA) aims to predict sentiment polarities of sentences with respect to given aspect categories. To detect the sentiment toward a particular aspect category in a sentence, most previous methods first generate an aspect category-specific sentence representation for the aspect category, then predict the sentiment polarity based on the representation. These methods ignore the fact that the sentiment of an aspect category mentioned in a sentence is an aggregation of the sentiments of the words indicating the aspect category in the sentence, which leads to suboptimal performance. In this paper, we propose a Multi-Instance Multi-Label Learning Network for Aspect-Category sentiment analysis (AC-MIMLLN), which treats sentences as bags, words as instances, and the words indicating an aspect category as the key instances of the aspect category. Given a sentence and the aspect categories mentioned in the sentence, AC-MIMLLN first predicts the sentiments of the instances, then finds the key instances for the aspect categories, finally obtains the sentiments of the sentence toward the aspect categories by aggregating the key instance sentiments. Experimental results on three public datasets demonstrate the effectiveness of AC-MIMLLN.
Learning to share by masking the non-shared for multi-domain sentiment classification

Yuan, Jianhua,Zhao, Yanyan,Qin, Bing

DOI: https://doi.org/10.1007/s13042-022-01556-0

2022-05-07

International Journal of Machine Learning and Cybernetics

Abstract:Multi-domain sentiment classification deals with the scenario where labeled data exists for multiple domains but is insufficient for training effective sentiment classifiers that work across domains. Thus, fully exploiting sentiment knowledge shared across domains is crucial for real-world applications. While many existing works try to extract domain-invariant features in high-dimensional space, such models fail to explicitly distinguish between shared and private features at the text level, which to some extent lacks interpretability. Based on the assumption that removing domain-related tokens from texts would help improve their domain invariance, we instead first transform original sentences to be domain-agnostic. To this end, we propose the BERTMasker model which explicitly masks domain-related words from texts, learns domain-invariant sentiment features from these domain-agnostic texts and uses those masked words to form domain-aware sentence representations. Empirical experiments on the benchmark multiple domain sentiment classification datasets demonstrate the effectiveness of our proposed model, which improves the accuracy on multi-domain and cross-domain settings by 1.91% and 3.31% respectively. Further analysis on masking proves that removing those domain-related and sentiment irrelevant tokens decreases texts’ domain separability, resulting in the performance degradation of a BERT-based domain classifier by over 12%.

computer science, artificial intelligence
Deep Learning for Sentiment Analysis : A Survey

Lei Zhang,Shuai Wang,Bing Liu

DOI: https://doi.org/10.48550/arXiv.1801.07883

2018-01-30

Abstract:Deep learning has emerged as a powerful machine learning technique that learns multiple layers of representations or features of the data and produces state-of-the-art prediction results. Along with the success of deep learning in many other application domains, deep learning is also popularly used in sentiment analysis in recent years. This paper first gives an overview of deep learning and then provides a comprehensive survey of its current applications in sentiment analysis.

Computation and Language,Information Retrieval,Machine Learning
Learning in Order! A Sequential Strategy to Learn Invariant Features for Multimodal Sentiment Analysis

Xianbing Zhao,Lizhen Qu,Tao Feng,Jianfei Cai,Buzhou Tang

DOI: https://doi.org/10.48550/arXiv.2409.04473

2024-09-05

Abstract:This work proposes a novel and simple sequential learning strategy to train models on videos and texts for multimodal sentiment analysis. To estimate sentiment polarities on unseen out-of-distribution data, we introduce a multimodal model that is trained either in a single source domain or multiple source domains using our learning strategy. This strategy starts with learning domain invariant features from text, followed by learning sparse domain-agnostic features from videos, assisted by the selected features learned in text. Our experimental results demonstrate that our model achieves significantly better performance than the state-of-the-art approaches on average in both single-source and multi-source settings. Our feature selection procedure favors the features that are independent to each other and are strongly correlated with their polarity labels. To facilitate research on this topic, the source code of this work will be publicly available upon acceptance.

Machine Learning,Artificial Intelligence
Domain adversarial learning for emotion recognition

Lian Zheng,Tao Jianhua,Liu Bin,Huang Jian

2019-01-01

Abstract: In practical applications for emotion recognition, users do not always exist in the training corpus. The mismatch between training speakers and testing speakers affects the performance of the trained model. To deal with this problem, we need our model to focus on emotion-related information, while ignoring the difference between speaker identities. In this paper, we look into the use of the domain adversarial neural network (DANN) to extract a common representation between different speakers. The primary task is to predict emotion labels. The secondary task is to learn a common representation where speaker identities can not be distinguished. By using the gradient reversal layer, the gradients coming from the secondary task are used to bring the representations for different speakers closer. To verify the effectiveness of the proposed method, we conduct experiments on the IEMOCAP database. Experimental results demonstrate that the proposed framework shows an absolute improvement of 3.48% over state-of-the-art strategies.
A Novel Scheme of Domain Transfer in Document-Level Cross-Domain Sentiment Classification.

Yueting Lei,Yanting Li

DOI: https://doi.org/10.1177/01655515211012329

2021-01-01

Journal of Information Science

Abstract:The sentiment classification aims to learn sentiment features from the annotated corpus and automatically predict the sentiment polarity of new sentiment text. However, people have different ways of expressing feelings in different domains. Thus, there are important differences in the characteristics of sentimental distribution across different domains. At the same time, in certain specific domains, due to the high cost of corpus collection, there is no annotated corpus available for the classification of sentiment. Therefore, it is necessary to leverage or reuse existing annotated corpus for training. In this article, we proposed a new algorithm for extracting central sentiment sentences in product reviews, and improved the pre-trained language model Bidirectional Encoder Representations from Transformers (BERT) to achieve the domain transfer for cross-domain sentiment classification. We used various pre-training language models to prove the effectiveness of the newly proposed joint algorithm for text-ranking and emotional words extraction, and utilised Amazon product reviews data set to demonstrate the effectiveness of our proposed domain-transfer framework. The experimental results of 12 different cross-domain pairs showed that the new cross-domain classification method was significantly better than several popular cross-domain sentiment classification methods.

Learning Domain Representation for Multi-Domain Sentiment Classification.

Domain Attention Model for Multi-Domain Sentiment Classification.

Multi-Domain Sentiment Classification Based on Domain-Aware Embedding and Attention

Domain adaptive learning for multi realm sentiment classification on big data

Collaborative Multi-domain Sentiment Classification

Incorporating Domain and Sentiment Supervision in Representation Learning for Domain Adaptation

Supervised Domain Adaptation for Sentiment Regression.

Domain-Aware Attention FC yDomain Feature Extractor and Classifier Sentiment Feature Extractor and ClassifierDomain-Aware Query VectorBiLSTM BiLSTM Domain-Aware Word Embedding

Cross-Lingual Sentiment Classification with Bilingual Document Representation Learning

Weighed Domain-Invariant Representation Learning for Cross-domain Sentiment Analysis

Collaboratively Training Sentiment Classifiers for Multiple Domains

Multi-domain active learning for text classification.

Learning Sentence Representations over Tree Structures for Target-Dependent Classification.

Improving Domain-Adapted Sentiment Classification by Deep Adversarial Mutual Learning

Emoji-Powered Representation Learning for Cross-Lingual Sentiment Classification

Multi-Instance Multi-Label Learning Networks for Aspect-Category Sentiment Analysis

Learning to share by masking the non-shared for multi-domain sentiment classification

Deep Learning for Sentiment Analysis : A Survey

Learning in Order! A Sequential Strategy to Learn Invariant Features for Multimodal Sentiment Analysis

Domain adversarial learning for emotion recognition

A Novel Scheme of Domain Transfer in Document-Level Cross-Domain Sentiment Classification.