The Natural Language Decathlon: Multitask Learning as Question Answering

Bryan McCann,Nitish Shirish Keskar,Caiming Xiong,Richard Socher

DOI: https://doi.org/10.48550/arXiv.1806.08730

2018-06-21

Abstract:Deep learning has improved performance on many natural language processing (NLP) tasks individually. However, general NLP models cannot emerge within a paradigm that focuses on the particularities of a single metric, dataset, and task. We introduce the Natural Language Decathlon (decaNLP), a challenge that spans ten tasks: question answering, machine translation, summarization, natural language inference, sentiment analysis, semantic role labeling, zero-shot relation extraction, goal-oriented dialogue, semantic parsing, and commonsense pronoun resolution. We cast all tasks as question answering over a context. Furthermore, we present a new Multitask Question Answering Network (MQAN) jointly learns all tasks in decaNLP without any task-specific modules or parameters in the multitask setting. MQAN shows improvements in transfer learning for machine translation and named entity recognition, domain adaptation for sentiment analysis and natural language inference, and zero-shot capabilities for text classification. We demonstrate that the MQAN's multi-pointer-generator decoder is key to this success and performance further improves with an anti-curriculum training strategy. Though designed for decaNLP, MQAN also achieves state of the art results on the WikiSQL semantic parsing task in the single-task setting. We also release code for procuring and processing data, training and evaluating models, and reproducing all experiments for decaNLP.

Computation and Language,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that although the performance of existing natural language processing (NLP) models has improved on specific tasks, these models lack generalization ability and cannot optimize multiple different NLP tasks simultaneously within one framework. To overcome this challenge, the author proposes "Natural Language Decathlon" (decaNLP), which is a challenge set covering ten NLP tasks, including but not limited to question answering, machine translation, summary generation, semantic parsing, sentiment analysis, natural language inference, etc. By redefining all tasks as context - based question - answering forms and introducing a Multitask Question Answering Network (MQAN), the author aims to explore a general model that can effectively handle multiple NLP tasks. MQAN can not only perform excellently in the multi - task environment of decaNLP, but also reach or exceed the existing state - of - the - art level in single - task settings, especially in the semantic parsing task. In addition, MQAN shows potential in transfer learning, domain adaptation and zero - sample learning, providing new directions and tools for research in the NLP field.