Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension
Bo Zheng,Haoyang Wen,Yaobo Liang,Nan Duan,Wanxiang Che,Daxin Jiang,Ming Zhou,Ting Liu
DOI: https://doi.org/10.18653/v1/2020.acl-main.599
2020-01-01
Abstract:Natural Questions is a new challenging machine reading comprehensionbenchmark with two-grained answers, which are a long answer (typically aparagraph) and a short answer (one or more entities inside the long answer).Despite the effectiveness of existing methods on this benchmark, they treatthese two sub-tasks individually during training while ignoring theirdependencies. To address this issue, we present a novel multi-grained machinereading comprehension framework that focuses on modeling documents at theirhierarchical nature, which are different levels of granularity: documents,paragraphs, sentences, and tokens. We utilize graph attention networks toobtain different levels of representations so that they can be learnedsimultaneously. The long and short answers can be extracted fromparagraph-level representation and token-level representation, respectively. Inthis way, we can model the dependencies between the two-grained answers toprovide evidence for each other. We jointly train the two sub-tasks, and ourexperiments show that our approach significantly outperforms previous systemsat both long and short answer criteria.