Abstract:Named entity recognition (NER) is frequently addressed as a sequence classification task where each input consists of one sentence of text. It is nevertheless clear that useful information for the task can often be found outside of the scope of a single-sentence context. Recently proposed self-attention models such as BERT can both efficiently capture long-distance relationships in input as well as represent inputs consisting of several sentences, creating new opportunitites for approaches that incorporate cross-sentence information in natural language processing tasks. In this paper, we present a systematic study exploring the use of cross-sentence information for NER using BERT models in five languages. We find that adding context in the form of additional sentences to BERT input systematically increases NER performance on all of the tested languages and models. Including multiple sentences in each input also allows us to study the predictions of the same sentences in different contexts. We propose a straightforward method, Contextual Majority Voting (CMV), to combine different predictions for sentences and demonstrate this to further increase NER performance with BERT. Our approach does not require any changes to the underlying BERT architecture, rather relying on restructuring examples for training and prediction. Evaluation on established datasets, including the CoNLL'02 and CoNLL'03 NER benchmarks, demonstrates that our proposed approach can improve on the state-of-the-art NER results on English, Dutch, and Finnish, achieves the best reported BERT-based results on German, and is on par with performance reported with other BERT-based approaches in Spanish. We release all methods implemented in this work under open licenses.

Split-NER: Named Entity Recognition via Two Question-Answering-based Classifications

S-NER: A Concise and Efficient Span-Based Model for Named Entity Recognition

Boundary Enhanced Neural Span Classification for Nested Named Entity Recognition

A Unified Generative Framework for Various NER Subtasks

Improving on Q & A Recurrent Neural Networks Using Noun-Tagging

Empowering Biomedical Named Entity Recognition Through Multi-Tagger Collaboration

Biomedical Named Entity Recognition at Scale

A Neural Span-Based Continual Named Entity Recognition Model

Win-Win Cooperation: Bundling Sequence and Span Models for Named Entity Recognition

SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition

Query-Based Named Entity Recognition

A Sequence-to-Set Network for Nested Named Entity Recognition.

Iterative Strategy for Named Entity Recognition with Imperfect Annotations.

Type-Aware Decomposed Framework for Few-Shot Named Entity Recognition

Learning to Reason: End-to-End Module Networks for Visual Question Answering

Hybrid NER System for Multi-Source Offer Feeds

Cross-domain NER under a Divide-and-Transfer Paradigm

SmartSpanNER: Making SpanNER Robust in Low Resource Scenarios

Exploring Cross-sentence Contexts for Named Entity Recognition with BERT

SpanNER: Named Entity Re-/Recognition As Span Prediction

Template-Based Named Entity Recognition Using BART