Abstract:Event detection refers to identifying event occurrences in a text and comprises of two subtasks; event identification and classification. We present EDM3, a novel approach for Event Detection that formulates three generative tasks: identification, classification, and combined detection. We show that EDM3 helps to learn transferable knowledge that can be leveraged to perform Event Detection and its subtasks concurrently, mitigating the error propagation inherent in pipelined approaches. Unlike previous dataset- or domain-specific approaches, EDM3 utilizes the existing knowledge of language models, allowing it to be trained over any classification schema. We evaluate EDM3 on multiple event detection datasets: RAMS, WikiEvents, MAVEN, and MLEE, showing that EDM3 outperforms 1) single-task performance by 8.4% on average and 2) multi-task performance without instructional prompts by 2.4% on average. We obtain SOTA results on RAMS (71.3% vs. 65.1% F-1) and competitive performance on other datasets. We analyze our approach to demonstrate its efficacy in low-resource and multi-sentence settings. We also show the effectiveness of this approach on non-standard event configurations such as multi-word and multi-class event triggers. Overall, our results show that EDM3 is a promising approach for Event Detection that has the potential for real-world applications.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is that in the Event Detection (ED) task, existing methods have deficiencies in identifying complex events, handling few - sample event detection scenarios, and cross - domain or cross - task adaptability. Specifically:
1. **Complex Event Identification**: Existing event detection models have difficulty in correctly identifying complex events.
2. **Few - Sample Event Detection**: In the case of a small amount of data, the performance of existing models is not good.
3. **Cross - Domain or Cross - Task Adaptability**: The trained models lack the ability to adapt across domains or tasks.
To solve these problems, the paper proposes a new method named EDM3, that is, to achieve joint training of event detection and its subtasks through multi - task text generation. EDM3 decomposes the event detection task into three generation tasks: Event Identification (EI), Event Classification (EC) and Combined Detection. In this way, the model can learn the knowledge transferred from subtasks, thereby improving the performance on the event detection task and reducing the error propagation in the pipeline method.
### Main Contributions
1. **Task Decomposition and Sequence Generation**: Transform event detection and its subtasks into sequence generation tasks, and use the knowledge transfer between subtasks to improve the performance of the complex main task (event detection).
2. **Performance across Datasets and Domains**: Achieve state - of - the - art or competitive performance on multiple datasets (such as RAMS, WikiEvents, MA VEN and MLEE).
3. **Effectiveness in Low - Resource and Complex Scenarios**: Verify the effectiveness of this method in complex real - world scenarios such as low - resource event scenarios, multi - word triggers and multi - class triggers through experiments.
### Experimental Results
- **RAMS Dataset**: EDM3 achieves an F1 score of 71.3% on the RAMS dataset, which is 6.2% higher than the previous best method GPTEDOT.
- **WikiEvents Dataset**: In the absence of existing baselines, EDM3 establishes a benchmark performance with an F1 score of 60.7%.
- **MA VEN Dataset**: Although the F1 score on the MA VEN dataset is 62.66%, which is lower than the existing best performance, the model performs relatively well on sparse categories.
- **MLEE Dataset**: On the biomedical domain dataset MLEE, EDM3 achieves an F1 score of 78.1%, which is comparable to more complex domain - specific methods.
### Analysis
- **Advantages of Multi - task Learning**: By simultaneously training event identification and classification tasks, the performance of EDM3 on all datasets is improved by at least 3%.
- **Diversity of Instruction Prompts**: Diverse instruction prompts can provide transferable knowledge, and even examples from different domains can improve the model performance.
- **Impact of Negative Samples**: Negative samples (no event instances) will affect the performance of event detection, especially in the WikiEvents dataset, the performance is significantly improved after removing negative samples.
- **Applicability in Low - Resource Scenarios**: EDM3 performs well in low - resource scenarios and can accurately identify events in a small number of labeled instances.
- **Identification of Multi - word Triggers and Multi - class Triggers**: EDM3 performs well in identifying multi - word triggers and multi - class triggers, with accuracies reaching 91% and 89% respectively.
In general, through the multi - task text generation method, EDM3 effectively solves the deficiencies of existing event detection methods and shows superior performance on multiple datasets and in complex scenarios.