Abstract:During the past decade, neural networks have become prominent in Natural Language Processing (NLP), notably for their capacity to learn relevant word representations from large unlabeled corpora. These word embeddings can then be transferred and finetuned for diverse end applications during a supervised training phase. More recently, in 2018, the transfer of entire pretrained Language Models and the preservation of their contextualization capacities enabled to reach unprecedented performance on virtually every NLP benchmark, sometimes even outperforming human baselines. However, as models reach such impressive scores, their comprehension abilities still appear as shallow, which reveal limitations of benchmarks to provide useful insights on their factors of performance and to accurately measure understanding capabilities. In this thesis, we study the behaviour of state-of-the-art models regarding generalization to facts unseen during training in two important Information Extraction tasks: Named Entity Recognition (NER) and Relation Extraction (RE). Indeed, traditional benchmarks present important lexical overlap between mentions and relations used for training and evaluating models, whereas the main interest of Information Extraction is to extract previously unknown information. We propose empirical studies to separate performance based on mention and relation overlap with the training set and find that pretrained Language Models are mainly beneficial to detect unseen mentions, in particular out-of-domain. While this makes them suited for real use cases, there is still a gap in performance between seen and unseen mentions that hurts generalization to new facts. In particular, even state-of-the-art ERE models rely on a shallow retention heuristic, basing their prediction more on arguments surface forms than context.

On Robustness and Bias Analysis of BERT-Based Relation Extraction

Can Fine-tuning Pre-trained Models Lead to Perfect NLP? A Study of the Generalizability of Relation Extraction.

Rescue Implicit and Long-tail Cases: Nearest Neighbor Relation Extraction

Relation Extraction with BERT-based Pre-trained Model.

Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study

Towards Integration of Discriminability and Robustness for Document-Level Relation Extraction

Siamese BERT Model with Adversarial Training for Relation Classification.

Relation Extraction as Open-book Examination: Retrieval-enhanced Prompt Tuning

Abstains from Prediction: Towards Robust Relation Extraction in Real World.

Active Testing: an Unbiased Evaluation Method for Distantly Supervised Relation Extraction.

Contextualization and Generalization in Entity and Relation Extraction

Relation Classification Via BERT with Piecewise Convolution and Focal Loss

The Exploration of the Reasoning Capability of BERT in Relation Extraction

A Brief Review of Relation Extraction Based on Pre-Trained Language Models

Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction

Minimally-Supervised Relation Induction from Pre-trained Language Model.

On the Robustness of Document-Level Relation Extraction Models to Entity Name Variations

A general approach for improving deep learning-based medical relation extraction using a pre-trained model and fine-tuning

Bias Modeling for Distantly Supervised Relation Extraction

Document-Level Relation Extraction with Distance-Dependent Bias Network and Neighbors Enhanced Loss.

A Dynamic Parameter Enhanced Network for Distant Supervised Relation Extraction