SUDA-Alibaba at MRP 2019: Graph-Based Models with BERT.

Yue Zhang,Wei Jiang,Qingrong Xia,Junjie Cao,Rui Wang,Zhenghua Li,Min Zhang

DOI: https://doi.org/10.18653/v1/k19-2014

2019-01-01

Abstract:Yue Zhang, Wei Jiang, Qingrong Xia, Junjie Cao, Rui Wang, Zhenghua Li, Min Zhang. Proceedings of the Shared Task on Cross-Framework Meaning Representation Parsing at the 2019 Conference on Natural Language Learning. 2019.

What problem does this paper attempt to address?

Prior-Bert and Multi-Task Learning for Target-Aspect-Sentiment Joint Detection

Cai Ke,Qingyu Xiong,Chao Wu,Zikai Liao,Hualing Yi

DOI: https://doi.org/10.1109/icassp43922.2022.9747904

2022-01-01

Abstract:Aspect-Based Sentiment Analysis (ABSA) is a fine-grained sentiment analysis task and has become a significant task with real-world scenario value. The challenge of this task is how to generate an effective text representation and construct an end-to-end model that can simultaneously detect (target, aspect, sentiment) triples from a sentence. Besides, the existing models do not take the heavily unbalanced distribution of labels into account and also do not give enough consideration to long-distance dependence of targets and aspect-sentiment pairs. To overcome these challenges, we propose a novel end-to-end model named Prior-BERT and Multi-Task Learning (PBERT-MTL), which can detect all triples more efficiently. We evaluate our model on SemEval-2015 and SemEval-2016 datasets. Extensive results show the validity of our work in this paper. In addition, our model also achieves higher performance on a series of subtasks of target-aspect-sentiment detection. Code is available at https://github.com/CQUPT-CaiKe/PBERT-MTL.
HIT-SCIR at MRP 2019: A Unified Pipeline for Meaning Representation Parsing Via Efficient Training and Effective Encoding.

Wanxiang Che,Longxu Dou,Yang Xu,Yuxuan Wang,Yijia Liu,Ting Liu

DOI: https://doi.org/10.18653/v1/k19-2007

2019-01-01

Abstract:Wanxiang Che, Longxu Dou, Yang Xu, Yuxuan Wang, Yijia Liu, Ting Liu. Proceedings of the Shared Task on Cross-Framework Meaning Representation Parsing at the 2019 Conference on Natural Language Learning. 2019.
SJTU at MRP 2019 - A Transition-Based Multi-Task Parser for Cross-Framework Meaning Representation Parsing.

Hongxiao Bai,Hai Zhao

DOI: https://doi.org/10.18653/v1/K19-2008

2019-01-01

Abstract:This paper describes the system of our team SJTU for our participation in the CoNLL 2019 Shared Task: Cross-Framework Meaning Representation Parsing. The goal of the task is to advance data-driven parsing into graph-structured representations of sentence meaning. This task includes five meaning representation frameworks: DM, PSD, EDS, UCCA, and AMR. These frameworks have different properties and structures. To tackle all the frameworks in one model, it is needed to find out the commonality of them. In our work, we define a set of the transition actions to once-for-all tackle all the frameworks and train a transition-based model to parse the meaning representation. The adopted multi-task model also can allow learning for one framework to benefit the others. In the final official evaluation of the shared task, our system achieves 42% F1 unified MRP metric score.
SJTU-NICT at MRP 2019 - Multi-Task Learning for End-to-End Uniform Semantic Graph Parsing.

Zuchao Li,Hai Zhao,Zhuosheng Zhang,Rui Wang,Masao Utiyama,Eiichiro Sumita

DOI: https://doi.org/10.18653/v1/K19-2004

2019-01-01

Abstract:This paper describes our SJTU-NICT’s system for participating in the shared task on Cross-Framework Meaning Representation Parsing (MRP) at the 2019 Conference for Computational Language Learning (CoNLL). Our system uses a graph-based approach to model a variety of semantic graph parsing tasks. Our main contributions in the submitted system are summarized as follows: 1. Our model is fully end-to-end and is capable of being trained only on the given training set which does not rely on any other extra training source including the companion data provided by the organizer; 2. We extend our graph pruning algorithm to a variety of semantic graphs, solving the problem of excessive semantic graph search space; 3. We introduce multi-task learning for multiple objectives within the same framework. The evaluation results show that our system achieved second place in the overall F1 score and achieved the best F1 score on the DM framework.
ShanghaiTech at MRP 2019: Sequence-to-Graph Transduction with Second-Order Edge Inference for Cross-Framework Meaning Representation Parsing

Xinyu Wang,Yixian Liu,Zixia Jia,Chengyue Jiang,Kewei Tu

DOI: https://doi.org/10.48550/arXiv.2004.03849

2020-04-08

Computation and Language

Abstract:This paper presents the system used in our submission to the \textit{CoNLL 2019 shared task: Cross-Framework Meaning Representation Parsing}. Our system is a graph-based parser which combines an extended pointer-generator network that generates nodes and a second-order mean field variational inference module that predicts edges. Our system achieved \nth{1} and \nth{2} place for the DM and PSD frameworks respectively on the in-framework ranks and achieved \nth{3} place for the DM framework on the cross-framework ranks.
Peking at MRP 2019: Factorization- and Composition-Based Parsing for Elementary Dependency Structures

Yufei Chen,Yajie Ye,Weiwei Sun

DOI: https://doi.org/10.18653/v1/k19-2016

2019-01-01

Abstract:We design, implement and evaluate two semantic parsers, which represent factorization- and composition-based approaches respectively, for Elementary Dependency Structures (EDS) at the CoNLL 2019 Shared Task on Cross-Framework Meaning Representation Parsing. The detailed evaluation of the two parsers gives us a new perception about parsing into linguistically enriched meaning representations: current neural EDS parsers are able to reach an accuracy at the inter-annotator agreement level in the same-epoch-and-domain setup.
Multi-Task Learning Model Based on BERT and Knowledge Graph for Aspect-Based Sentiment Analysis

Zhu He,Honglei Wang,Xiaoping Zhang

DOI: https://doi.org/10.3390/electronics12030737

IF: 2.9

2023-02-02

Electronics

Abstract:Aspect-based sentiment analysis (ABSA) aims to identify the sentiment of an aspect in a given sentence and thus can provide people with comprehensive information. However, many conventional methods need help to discover the linguistic knowledge implicit in sentences. Additionally, they are susceptible to unrelated words. To improve the performance of the model in the ABSA task, a multi-task sentiment analysis model based on Bidirectional Encoder Representation from Transformers (BERT) and a Knowledge Graph (SABKG) is proposed in this paper. Expressly, part-of-speech information is incorporated into the output representation of BERT, thereby obtaining textual semantic information through linguistic knowledge. It also enhances the textual representation to identify the aspect terms. Moreover, this paper constructs a knowledge graph of aspect and sentiment words. It uses a graph neural network to learn the embeddings in the triplet of "aspect word, sentiment polarity, sentiment word". The constructed graph improves the contextual relationship between the text's aspect and sentiment words. The experimental results on three open datasets show that the proposed model can achieve the most advanced performance compared with previous models.

engineering, electrical & electronic,computer science, information systems,physics, applied
HLT@SUDA at SemEval 2019 Task 1: UCCA Graph Parsing as Constituent Tree Parsing.

Wei Jiang,Yu Zhang,Zhenghua Li,Min Zhang

DOI: https://doi.org/10.18653/v1/s19-2002

2019-01-01

Abstract:This paper describes a simple UCCA semantic graph parsing approach. The key idea is to convert a UCCA semantic graph into a constituent tree, in which extra labels are deliberately designed to mark remote edges and discontinuous nodes for future recovery. In this way, we can make use of existing syntactic parsing techniques. Based on the data statistics, we recover discontinuous nodes directly according to the output labels of the constituent parser and use a biaffine classification model to recover the more complex remote edges. The classification model and the constituent parser are simultaneously trained under the multi-task learning framework. We use the multilingual BERT as extra features in the open tracks. Our system ranks the first place in the six English/German closed/open tracks among seven participating systems. For the seventh cross-lingual track, where there is little training data for French, we propose a language embedding approach to utilize English and German training data, and our result ranks the second place.
TSMind: Alibaba and Soochow University's Submission to the WMT22 Translation Suggestion Task

Xin Ge,Ke Wang,Jiayi Wang,Nini Xiao,Xiangyu Duan,Yu Zhao,Yuqi Zhang

DOI: https://doi.org/10.48550/arXiv.2211.08987

2022-11-16

Abstract:This paper describes the joint submission of Alibaba and Soochow University, TSMind, to the WMT 2022 Shared Task on Translation Suggestion (TS). We participate in the English-German and English-Chinese tasks. Basically, we utilize the model paradigm fine-tuning on the downstream tasks based on large-scale pre-trained models, which has recently achieved great success. We choose FAIR's WMT19 English-German news translation system and MBART50 for English-Chinese as our pre-trained models. Considering the task's condition of limited use of training data, we follow the data augmentation strategies proposed by WeTS to boost our TS model performance. The difference is that we further involve the dual conditional cross-entropy model and GPT-2 language model to filter augmented data. The leader board finally shows that our submissions are ranked first in three of four language directions in the Naive TS task of the WMT22 Translation Suggestion task.

Computation and Language
Semantics-aware BERT for Language Understanding.

Zhuosheng Zhang,Yuwei Wu,Hai Zhao,Zuchao Li,Shuailiang Zhang,Xi Zhou,Xiang Zhou

DOI: https://doi.org/10.1609/aaai.v34i05.6510

2020-01-01

Proceedings of the AAAI Conference on Artificial Intelligence

Abstract:The latest work on language representations carefully integrates contextualized features into language model training, which enables a series of success especially in various machine reading comprehension and natural language inference tasks. However, the existing language representation models including ELMo, GPT and BERT only exploit plain context-sensitive features such as character or word embeddings. They rarely consider incorporating structured semantic information which can provide rich semantics for language representation. To promote natural language understanding, we propose to incorporate explicit contextual semantics from pre-trained semantic role labeling, and introduce an improved language representation model, Semantics-aware BERT (SemBERT), which is capable of explicitly absorbing contextual semantics over a BERT backbone. SemBERT keeps the convenient usability of its BERT precursor in a light fine-tuning way without substantial task-specific modifications. Compared with BERT, semantics-aware BERT is as simple in concept but more powerful. It obtains new state-of-the-art or substantially improves results on ten reading comprehension and language inference tasks.
Aspect-Based Sentiment Analysis using Local Context Focus Mechanism with DeBERTa

Tianyu Zhao,Junping Du,Zhe Xue,Ang Li,Zeli Guan

DOI: https://doi.org/10.48550/arXiv.2207.02424

2022-07-06

Computation and Language

Abstract:Text sentiment analysis, also known as opinion mining, is research on the calculation of people's views, evaluations, attitude and emotions expressed by entities. Text sentiment analysis can be divided into text-level sentiment analysis, sen-tence-level sentiment analysis and aspect-level sentiment analysis. Aspect-Based Sentiment Analysis (ABSA) is a fine-grained task in the field of sentiment analysis, which aims to predict the polarity of aspects. The research of pre-training neural model has significantly improved the performance of many natural language processing tasks. In recent years, pre training model (PTM) has been applied in ABSA. Therefore, there has been a question, which is whether PTMs contain sufficient syntactic information for ABSA. In this paper, we explored the recent DeBERTa model (Decoding-enhanced BERT with disentangled attention) to solve Aspect-Based Sentiment Analysis problem. DeBERTa is a kind of neural language model based on transformer, which uses self-supervised learning to pre-train on a large number of original text corpora. Based on the Local Context Focus (LCF) mechanism, by integrating DeBERTa model, we purpose a multi-task learning model for aspect-based sentiment analysis. The experiments result on the most commonly used the laptop and restaurant datasets of SemEval-2014 and the ACL twitter dataset show that LCF mechanism with DeBERTa has significant improvement.
BURT: BERT-inspired Universal Representation from Learning Meaningful Segment

Yian Li,Hai Zhao

2020-01-01

Abstract:Although pre-trained contextualized language models such as BERT achieve significant performance on various downstream tasks, current language representation focuses on linguistic objective at a specific granularity. Thus this work introduces the universal representation learning, i.e., embeddings of different levels of linguistic unit in a uniform vector space. We present a universal representation model, BURT (BERT-inspired Universal Representation from learning meaningful segmenT), to encode different levels of linguistic unit into the same vector space. Specifically, we extract meaningful segments based on point-wise mutual information (PMI) to incorporate different granular objectives into the pre-training stage. We conduct experiments on datasets for English and Chinese including the GLUE and CLUE benchmarks, where our model surpasses its baselines and alternatives on a wide range of downstream tasks. We present our approach of constructing analogy datasets in terms of words, phrases and sentences and experiment with multiple representation models to examine geometric properties of the learned vector space through a task-independent evaluation. Finally, we verify the effectiveness of our method in two real-world text matching scenarios. As a result, our model significantly outperforms existing information retrieval (IR) methods and yields universal representations that can be directly applied to retrieval-based question-answering and natural language
Aspect-Level Sentiment Analysis Approach via BERT and Aspect Feature Location Model

Guangyao Pang,Keda Lu,Xiaoying Zhu,Jie He,Zhiyi Mo,Zizhen Peng,Baoxing Pu

DOI: https://doi.org/10.1155/2021/5534615

2021-08-06

Wireless Communications and Mobile Computing

Abstract:With the rapid development of Internet social platforms, buyer shows (such as comment text) have become an important basis for consumers to understand products and purchase decisions. The early sentiment analysis methods were mainly text-level and sentence-level, which believed that a text had only one sentiment. This phenomenon will cover up the details, and it is difficult to reflect people’s fine-grained and comprehensive sentiments fully, leading to people’s wrong decisions. Obviously, aspect-level sentiment analysis can obtain a more comprehensive sentiment classification by mining the sentiment tendencies of different aspects in the comment text. However, the existing aspect-level sentiment analysis methods mainly focus on attention mechanism and recurrent neural network. They lack emotional sensitivity to the position of aspect words and tend to ignore long-term dependencies. In order to solve this problem, on the basis of Bidirectional Encoder Representations from Transformers (BERT), this paper proposes an effective aspect-level sentiment analysis approach (ALM-BERT) by constructing an aspect feature location model. Specifically, we use the pretrained BERT model first to mine more aspect-level auxiliary information from the comment context. Secondly, for the sake of learning the expression features of aspect words and the interactive information of aspect words’ context, we construct an aspect-based sentiment feature extraction method. Finally, we construct evaluation experiments on three benchmark datasets. The experimental results show that the aspect-level sentiment analysis performance of the ALM-BERT approach proposed in this paper is significantly better than other comparison methods.

computer science, information systems,telecommunications,engineering, electrical & electronic
XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representations

Yusen Zhang,Jun Wang,Zhiguo Wang,Rui Zhang

2023-06-07

Abstract:Cross-Lingual Semantic Parsing (CLSP) aims to translate queries in multiple natural languages (NLs) into meaning representations (MRs) such as SQL, lambda calculus, and logic forms. However, existing CLSP models are separately proposed and evaluated on datasets of limited tasks and applications, impeding a comprehensive and unified evaluation of CLSP on a diverse range of NLs and MRs. To this end, we present XSemPLR, a unified benchmark for cross-lingual semantic parsing featured with 22 natural languages and 8 meaning representations by examining and selecting 9 existing datasets to cover 5 tasks and 164 domains. We use XSemPLR to conduct a comprehensive benchmark study on a wide range of multilingual language models including encoder-based models (mBERT, XLM-R), encoder-decoder models (mBART, mT5), and decoder-based models (Codex, BLOOM). We design 6 experiment settings covering various lingual combinations (monolingual, multilingual, cross-lingual) and numbers of learning samples (full dataset, few-shot, and zero-shot). Our experiments show that encoder-decoder models (mT5) achieve the highest performance compared with other popular models, and multilingual training can further improve the average performance. Notably, multilingual large language models (e.g., BLOOM) are still inadequate to perform CLSP tasks. We also find that the performance gap between monolingual training and cross-lingual transfer learning is still significant for multilingual models, though it can be mitigated by cross-lingual few-shot training. Our dataset and code are available at <a class="link-external link-https" href="https://github.com/psunlpgroup/XSemPLR" rel="external noopener nofollow">this https URL</a>.

Computation and Language
CAT-BERT: A Context-Aware Transferable BERT Model for Multi-turn Machine Reading Comprehension.

Cen Chen,Xinjing Huang,Feng Ji,Chengyu Wang,Minghui Qiu,Jun Huang,Yin Zhang

DOI: https://doi.org/10.1007/978-3-030-73197-7_10

2021-01-01

Abstract:Machine Reading Comprehension (MRC) is an important NLP task with the goal of extracting answers to user questions from background passages. For conversational applications, modeling the contexts under the multi-turn setting is highly necessary for MRC, which has drawn great attention recently. Past studies on multi-turn MRC usually focus on a single domain, ignoring the fact that knowledge in different MRC tasks are transferable. To address this issue, we present a unified framework to model both single-turn and multi-turn MRC tasks which allows knowledge sharing from different source MRC tasks to help solve the target MRC task. Specifically, the Context-Aware Transferable Bidirectional Encoder Representations from Transformers (CAT-BERT) model is proposed, which jointly learns to solve both single-turn and multi-turn MRC tasks in a single pre-trained language model. In this model, both history questions and answers are encoded into the contexts for the multi-turn setting. To capture the task-level importance of different layer outputs, a task-specific attention layer is further added to the CAT-BERT outputs, reflecting the positions that the model should pay attention to for a specific MRC task. Extensive experimental results and ablation studies show that CAT-BERT achieves competitive results in multi-turn MRC tasks, outperforming strong baselines.
MACT: Model-Agnostic Cross-Lingual Training for Discourse Representation Structure Parsing

Jiangming Liu

2024-06-03

Abstract:Discourse Representation Structure (DRS) is an innovative semantic representation designed to capture the meaning of texts with arbitrary lengths across languages. The semantic representation parsing is essential for achieving natural language understanding through logical forms. Nevertheless, the performance of DRS parsing models remains constrained when trained exclusively on monolingual data. To tackle this issue, we introduce a cross-lingual training strategy. The proposed method is model-agnostic yet highly effective. It leverages cross-lingual training data and fully exploits the alignments between languages encoded in pre-trained language models. The experiments conducted on the standard benchmarks demonstrate that models trained using the cross-lingual training method exhibit significant improvements in DRS clause and graph parsing in English, German, Italian and Dutch. Comparing our final models to previous works, we achieve state-of-the-art results in the standard benchmarks. Furthermore, the detailed analysis provides deep insights into the performance of the parsers, offering inspiration for future research in DRS parsing. We keep updating new results on benchmarks to the appendix.

Computation and Language
A Joint Training Dual-MRC Framework for Aspect Based Sentiment Analysis

Yue Mao,Yi Shen,Chao Yu,Longjun Cai

DOI: https://doi.org/10.48550/arXiv.2101.00816

2021-04-07

Abstract:Aspect based sentiment analysis (ABSA) involves three fundamental subtasks: aspect term extraction, opinion term extraction, and aspect-level sentiment classification. Early works only focused on solving one of these subtasks individually. Some recent work focused on solving a combination of two subtasks, e.g., extracting aspect terms along with sentiment polarities or extracting the aspect and opinion terms pair-wisely. More recently, the triple extraction task has been proposed, i.e., extracting the (aspect term, opinion term, sentiment polarity) triples from a sentence. However, previous approaches fail to solve all subtasks in a unified end-to-end framework. In this paper, we propose a complete solution for ABSA. We construct two machine reading comprehension (MRC) problems and solve all subtasks by joint training two BERT-MRC models with parameters sharing. We conduct experiments on these subtasks, and results on several benchmark datasets demonstrate the effectiveness of our proposed framework, which significantly outperforms existing state-of-the-art methods.

Computation and Language,Artificial Intelligence
Multi-Grained Attention Representation With ALBERT for Aspect-Level Sentiment Classification

Yuezhe Chen,Lingyun Kong,Yang Wang,Dezhi Kong

DOI: https://doi.org/10.1109/access.2021.3100299

IF: 3.9

2021-01-01

IEEE Access

Abstract:Aspect-level sentiment classification aims to solve the problem, which is to judge the sentiment tendency of each aspect in a sentence with multiple aspects. Previous works mainly employed Long Short-Term Memory (LSTM) and Attention mechanisms to fuse information between aspects and sentences, or to improve large language models such as BERT to adapt aspect-level sentiment classification tasks. The former methods either did not integrate the interactive information of related aspects and sentences, or ignored the feature extraction of sentences. This paper proposes a novel multi-grained attention representation with ALBERT (MGAR-ALBERT). It can learn the representation that contains the relevant information of the sentence and the aspect, while integrating it into the process of sentence modeling with multi granularity, and finally get a comprehensive sentence representation. In Masked LM (MLM) task, in order to avoid the influence of aspect words being masked in the initial stage of the pre-training, the noise linear cosine decay is introduced into <span class="mjpage"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="9.67ex" height="2.343ex" style="vertical-align: -0.671ex;" viewBox="0 -719.6 4163.4 1008.6" role="img" focusable="false" xmlns="http://www.w3.org/2000/svg"><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"> <use xlink:href="#MJMATHI-6E" x="0" y="0"></use> <use xlink:href="#MJMAIN-2212" x="822" y="0"></use> <use xlink:href="#MJMATHI-67" x="1823" y="0"></use> <use xlink:href="#MJMATHI-72" x="2303" y="0"></use> <use xlink:href="#MJMATHI-61" x="2755" y="0"></use> <use xlink:href="#MJMATHI-6D" x="3284" y="0"></use></g></svg></span> . We implemented a series of comparative experiments to verify the effectiveness of the method. The experimental results show that our model can achieve excellent results on Restaurant dataset with numerous number of parameters reduced, and it is not inferior to other models on Laptop dataset.<svg xmlns="http://www.w3.org/2000/svg" style="display: none;"><defs id="MathJax_SVG_glyphs"><path stroke-width="1" id="MJMATHI-6E" d="M21 287Q22 293 24 303T36 341T56 388T89 425T135 442Q171 442 195 424T225 390T231 369Q231 367 232 367L243 378Q304 442 382 442Q436 442 469 415T503 336T465 179T427 52Q427 26 444 26Q450 26 453 27Q482 32 505 65T540 145Q542 153 560 153Q580 153 580 145Q580 144 576 130Q568 101 554 73T508 17T439 -10Q392 -10 371 17T350 73Q350 92 386 193T423 345Q423 404 379 404H374Q288 404 229 303L222 291L189 157Q156 26 151 16Q138 -11 108 -11Q95 -11 87 -5T76 7T74 17Q74 30 112 180T152 343Q153 348 153 366Q153 405 129 405Q91 405 66 305Q60 285 60 284Q58 278 41 278H27Q21 284 21 287Z"></path><path stroke-width="1" id="MJMAIN-2212" d="M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z"></path><path stroke-width="1" id="MJMATHI-67" d="M311 43Q296 30 267 15T206 0Q143 0 105 45T66 160Q66 265 143 353T314 442Q361 442 401 394L404 398Q406 401 409 404T418 412T431 419T447 422Q461 422 470 413T480 394Q480 379 423 152T363 -80Q345 -134 286 -169T151 -205Q10 -205 10 -137Q10 -111 28 -91T74 -71Q89 -71 102 -80T116 -111Q116 -121 114 -130T107 -144T99 -154T92 -162L90 -164H91Q101 -167 151 -167Q189 -167 211 -155Q234 -144 254 -122T282 -75Q288 -56 298 -13Q311 35 311 43ZM384 328L380 339Q377 350 375 354T369 368T359 382T346 393T328 402T306 405Q262 405 221 352Q191 313 171 233T151 117Q151 38 213 38Q269 38 323 108L331 118L384 328Z"></path><path stroke-width="1" id="MJMATHI-72" d="M21 287Q22 290 23 295T28 317T38 348T53 381T73 411T99 433T132 442Q161 442 183 430T214 408T225 388Q227 382 228 382T236 389Q284 441 347 441H350Q398 441 422 400Q430 381 430 363Q430 333 417 315T391 292T366 288Q346 288 334 299T322 328Q322 376 378 392Q356 405 342 405Q286 405 239 331Q229 315 224 298T190 165Q156 25 151 16Q138 -11 108 -11Q95 -11 87 -5T76 7T74 17Q74 30 114 189T154 366Q154 405 128 405Q107 405 92 377T68 316T57 280Q55 278 41 278H27Q21 284 21 287Z"></path><path stroke-width="1" id="MJMATHI-61" d="M33 157Q33 258 109 349T280 441Q331 441 370 392Q386 422 416 422Q429 422 439 414T449 394Q449 381 412 234T374 68Q374 43 381 35T402 26Q411 27 422 35Q443 55 463 131Q469 151 473 152Q475 153 483 153H487Q506 153 506 144Q506 138 501 117T481 63T449 13Q436 0 417 -8Q409 -10 393 -10Q359 -10 336 5T306 36L300 51Q299 52 296 50Q294 48 292 46Q233 -10 172 -10Q117 -10 75 30T33 157ZM351 328Q351 334 346 350T323 385T277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q217 26 254 59T298 110Q300 114 325 217T351 328Z"></path><path stroke-width="1" id="MJMATHI-6D" d="M21 287Q22 293 24 303T36 341T56 388T88 425T132 442T175 435T205 417T221 395T229 376L231 369Q231 367 232 367L243 378Q303 442 384 442Q401 442 415 440T441 433T460 423T475 411T485 398T493 385T497 373T500 364T502 357L510 367Q573 442 659 442Q713 442 746 415T780 336Q780 285 742 178T704 50Q705 36 709 31T724 26Q752 26 776 56T815 138Q818 149 821 151T837 153Q857 153 857 145Q857 144 853 130Q845 101 831 73T785 17T716 -10Q669 -10 648 17T627 73Q627 92 663 193T700 345Q700 404 656 404H651Q565 404 506 303L499 291L466 157Q433 26 428 16Q415 -11 385 -11Q372 -11 364 -4T353 8T350 18Q350 29 384 161L420 307Q423 322 423 345Q423 404 379 404H374Q288 404 229 303L222 291L189 157Q156 26 151 16Q138 -11 108 -11Q95 -11 87 -5T76 7T74 17Q74 30 112 181Q151 335 151 342Q154 357 154 369Q154 405 129 405Q107 405 92 377T69 316T57 280Q55 278 41 278H27Q21 284 21 287Z"></path></defs></svg>

computer science, information systems,telecommunications,engineering, electrical & electronic
A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing

Hang Yan,Xipeng Qiu,Xuanjing Huang

DOI: https://doi.org/10.48550/arXiv.1904.04697

2019-12-18

Abstract:Chinese word segmentation and dependency parsing are two fundamental tasks for Chinese natural language processing. The dependency parsing is defined on word-level. Therefore word segmentation is the precondition of dependency parsing, which makes dependency parsing suffer from error propagation and unable to directly make use of the character-level pre-trained language model (such as BERT). In this paper, we propose a graph-based model to integrate Chinese word segmentation and dependency parsing. Different from previous transition-based joint models, our proposed model is more concise, which results in fewer efforts of feature engineering. Our graph-based joint model achieves better performance than previous joint models and state-of-the-art results in both Chinese word segmentation and dependency parsing. Besides, when BERT is combined, our model can substantially reduce the performance gap of dependency parsing between joint models and gold-segmented word-based models. Our code is publicly available at <a class="link-external link-https" href="https://github.com/fastnlp/JointCwsParser" rel="external noopener nofollow">this https URL</a>.

Computation and Language,Artificial Intelligence
Explorers at #SMM4H 2023: Enhancing BERT for Health Applications through Knowledge and Model Fusion

Xutong Yue,Xilai Wang,Yuxin He,Zhenkun Zhou

2023-12-17

Abstract:An increasing number of individuals are willing to post states and opinions in social media, which has become a valuable data resource for studying human health. Furthermore, social media has been a crucial research point for healthcare now. This paper outlines the methods in our participation in the #SMM4H 2023 Shared Tasks, including data preprocessing, continual pre-training and fine-tuned optimization strategies. Especially for the Named Entity Recognition (NER) task, we utilize the model architecture named W2NER that effectively enhances the model generalization ability. Our method achieved first place in the Task 3. This paper has been peer-reviewed and accepted for presentation at the #SMM4H 2023 Workshop.

Computation and Language,Social and Information Networks

SUDA-Alibaba at MRP 2019: Graph-Based Models with BERT.

Prior-Bert and Multi-Task Learning for Target-Aspect-Sentiment Joint Detection

HIT-SCIR at MRP 2019: A Unified Pipeline for Meaning Representation Parsing Via Efficient Training and Effective Encoding.

SJTU at MRP 2019 - A Transition-Based Multi-Task Parser for Cross-Framework Meaning Representation Parsing.

SJTU-NICT at MRP 2019 - Multi-Task Learning for End-to-End Uniform Semantic Graph Parsing.

ShanghaiTech at MRP 2019: Sequence-to-Graph Transduction with Second-Order Edge Inference for Cross-Framework Meaning Representation Parsing

Peking at MRP 2019: Factorization- and Composition-Based Parsing for Elementary Dependency Structures

Multi-Task Learning Model Based on BERT and Knowledge Graph for Aspect-Based Sentiment Analysis

HLT@SUDA at SemEval 2019 Task 1: UCCA Graph Parsing as Constituent Tree Parsing.

TSMind: Alibaba and Soochow University's Submission to the WMT22 Translation Suggestion Task

Semantics-aware BERT for Language Understanding.

Aspect-Based Sentiment Analysis using Local Context Focus Mechanism with DeBERTa

BURT: BERT-inspired Universal Representation from Learning Meaningful Segment

Aspect-Level Sentiment Analysis Approach via BERT and Aspect Feature Location Model

XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representations

CAT-BERT: A Context-Aware Transferable BERT Model for Multi-turn Machine Reading Comprehension.

MACT: Model-Agnostic Cross-Lingual Training for Discourse Representation Structure Parsing

A Joint Training Dual-MRC Framework for Aspect Based Sentiment Analysis

Multi-Grained Attention Representation With ALBERT for Aspect-Level Sentiment Classification

A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing

Explorers at #SMM4H 2023: Enhancing BERT for Health Applications through Knowledge and Model Fusion