Abstract:Named entity recognition (NER) is a fundamental task in natural language processing that involves identifying and classifying entities in sentences into pre-defined types. It plays a crucial role in various research fields, including entity linking, question answering, and online product recommendation. Recent studies have shown that incorporating multilingual and multimodal datasets can enhance the effectiveness of NER. This is due to language transfer learning and the presence of shared implicit features across different modalities. However, the lack of a dataset that combines multilingualism and multimodality has hindered research exploring the combination of these two aspects, as multimodality can help NER in multiple languages simultaneously. In this paper, we aim to address a more challenging task: multilingual and multimodal named entity recognition (MMNER), considering its potential value and influence. Specifically, we construct a large-scale MMNER dataset with four languages (English, French, German and Spanish) and two modalities (text and image). To tackle this challenging MMNER task on the dataset, we introduce a new model called 2M-NER , which aligns the text and image representations using contrastive learning and integrates a multimodal collaboration module to effectively depict the interactions between the two modalities. Extensive experimental results demonstrate that our model achieves the highest F1 score in multilingual and multimodal NER tasks compared to some comparative and representative baselines. Additionally, in a challenging analysis, we discovered that sentence-level alignment interferes a lot with NER models, indicating the higher level of difficulty in our dataset.

MultiCoNER: A Large-scale Multilingual Dataset for Complex Named Entity Recognition

MultiCoNER v2: a Large Multilingual dataset for Fine-grained and Noisy Named Entity Recognition

NERetrieve: Dataset for Next Generation Named Entity Recognition and Retrieval

An Open-Source Dataset and A Multi-Task Model for Malay Named Entity Recognition

SemEval-2023 Task 2: Fine-grained Multilingual Named Entity Recognition (MultiCoNER 2)

DAMO-NLP at SemEval-2023 Task 2: A Unified Retrieval-augmented System for Multilingual Named Entity Recognition

Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition

Automatically Building Large-Scale Named Entity Recognition Corpora from Chinese Wikipedia

2M-NER: contrastive learning for multilingual and multimodal NER with language and modal fusion

Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmark

Towards Malay named entity recognition: an open-source dataset and a multi-task framework

SocialNER2.0: A comprehensive dataset for enhancing named entity recognition in short human-produced text

GEIC: Universal and Multilingual Named Entity Recognition with Large Language Models

DistALANER: Distantly Supervised Active Learning Augmented Named Entity Recognition in the Open Source Software Ecosystem

MSNER: A Multilingual Speech Dataset for Named Entity Recognition

M-CNER: A Corpus for Chinese Named Entity Recognition in Multi-Domains.

CoLaDa: A Collaborative Label Denoising Framework for Cross-lingual Named Entity Recognition.

Cross-Lingual Named Entity Recognition Using Parallel Corpus: A New Approach Using XLM-RoBERTa Alignment

AgCNER, the First Large-Scale Chinese Named Entity Recognition Dataset for Agricultural Diseases and Pests

NERO: a biomedical named-entity (recognition) ontology with a large, annotated corpus reveals meaningful associations through text embedding