Abstract:Information extraction (IE) is a fundamental area in natural language processing where prompting large language models (LLMs), even with in-context examples, cannot defeat small LMs tuned on very small IE datasets. We observe that IE tasks, such as named entity recognition and relation extraction, all focus on extracting important information, which can be formalized as a label-to-span matching. In this paper, we propose a novel framework MetaIE to build a small LM as meta-model by learning to extract "important information", i.e., the meta-understanding of IE, so that this meta-model can be adapted to all kind of IE tasks effectively and efficiently. Specifically, MetaIE obtains the small LM via a symbolic distillation from an LLM following the label-to-span scheme. We construct the distillation dataset via sampling sentences from language model pre-training datasets (e.g., OpenWebText in our implementation) and prompting an LLM to identify the typed spans of "important information". We evaluate the meta-model under the few-shot adaptation setting. Extensive results on 13 datasets from 6 IE tasks confirm that MetaIE can offer a better starting point for few-shot tuning on IE datasets and outperform other meta-models from (1) vanilla language model pre-training, (2) multi-IE-task pre-training with human annotations, and (3) single-IE-task symbolic distillation from LLM. Moreover, we provide comprehensive analyses of MetaIE, such as the size of the distillation dataset, the meta-model architecture, and the size of the meta-model.

IELM: an Open Information Extraction Benchmark for Pre-Trained Language Models

Mastering the Task of Open Information Extraction with Large Language Models and Consistent Reasoning Environment

A Survey on Open Information Extraction from Rule-based Model to Large Language Model (meta)

Efficient Data Learning for Open Information Extraction with Pre-trained Language Models

IEPile: Unearthing Large Scale Schema-Conditioned Information Extraction Corpus

Assessing the Performance of Chinese Open Source Large Language Models in Information Extraction Tasks

PIVOINE: Instruction Tuning for Open-world Information Extraction

Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study

Leveraging Linguistically Enhanced Embeddings for Open Information Extraction

AnnIE: An Annotation Platform for Constructing Complete Open Information Extraction Benchmark

A Survey on Neural Open Information Extraction: Current Status and Future Directions

Rules still work for Open Information Extraction

$\textit{BenchIE}^{FL}$ : A Manually Re-Annotated Fact-Based Open Information Extraction Benchmark

RUIE: Retrieval-based Unified Information Extraction using Large Language Model

IDOL: Indicator-oriented Logic Pre-training for Logical Reasoning

GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-Distribution Generalization Perspective.

milIE: Modular & Iterative Multilingual Open Information Extraction

An Empirical Study on Information Extraction using Large Language Models

Large Language Models for Generative Information Extraction: A Survey

MetaIE: Distilling a Meta Model from LLM for All Kinds of Information Extraction Tasks