Adaptive Knowledge-Enhanced Bayesian Meta-Learning for Few-shot Event Detection

Shirong Shen,Tongtong Wu,Guilin Qi,Yuan-Fang Li,Gholamreza Haffari,Sheng Bi
DOI: https://doi.org/10.48550/arXiv.2105.09509
2021-05-30
Abstract:Event detection (ED) aims at detecting event trigger words in sentences and classifying them into specific event types. In real-world applications, ED typically does not have sufficient labelled data, thus can be formulated as a few-shot learning problem. To tackle the issue of low sample diversity in few-shot ED, we propose a novel knowledge-based few-shot event detection method which uses a definition-based encoder to introduce external event knowledge as the knowledge prior of event types. Furthermore, as external knowledge typically provides limited and imperfect coverage of event types, we introduce an adaptive knowledge-enhanced Bayesian meta-learning method to dynamically adjust the knowledge prior of event types. Experiments show our method consistently and substantially outperforms a number of baselines by at least 15 absolute F1 points under the same few-shot settings.
Computation and Language,Artificial Intelligence
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the few - shot learning challenges faced by event detection (ED) in practical applications. Specifically, the event detection task aims to identify trigger words from text and classify them into specific event types. However, in real - world applications, there is usually a lack of sufficient labeled data, which makes event detection can be modeled as a few - shot learning problem. ### Specific manifestations of the problem 1. **Insufficient sample diversity**: In few - shot event detection, the number of samples in the support set is limited and lacks diversity, causing the model to be difficult to generalize to new event types. 2. **Incomplete coverage and uncertainty of external knowledge**: Introducing external knowledge (such as FrameNet) can enhance the performance of the model, but these knowledge bases usually have incomplete coverage of event types, and there may be misalignment between the predefined knowledge and the actual application scenarios. ### The method proposed in the paper To solve the above problems, the paper proposes an Adaptive Knowledge - Enhanced Bayesian Meta - Learning framework (AKE - BML). The main innovations of this method include: 1. **Define the encoder**: Introduce external event knowledge as prior knowledge of event types by defining an encoder. 2. **Adaptive knowledge enhancement**: Introduce an adaptive knowledge - enhanced Bayesian meta - learning method to dynamically adjust the prior knowledge of event types. 3. **Generation of prototype vectors**: Use the support set and external knowledge to construct prototype vectors of event types, and optimize the model parameters through Monte Carlo sampling and Stochastic Gradient Langevin Dynamics. ### Core formulas of the method - **Knowledge encoding**: \[ h_t = f_h([e^*_A; e^*_L]) \] where \(e^*_A\) and \(e^*_L\) are the argument prior encoding and trigger word prior encoding generated by the attention mechanism respectively. - **Knowledge offset**: \[ \Delta h_t=\lambda_t\odot(m_t - h_t) \] where \(\lambda_t\) is an adaptive weight used to balance the influence of the support set and prior knowledge. - **Posterior distribution**: \[ p(V_{T_S}|X_S, Y_S, F)\propto p(Y_S|X_S, V_{T_S})p(V_{T_S}|F) \] - **Prior distribution of prototype vectors**: \[ p(V_{T_S}|F)=\prod_{t\in T_S}N(v_t|h_t+\Delta h_t, I) \] ### Experimental results The experimental results show that in all six N - way - M - shot settings, the F1 score of this method is at least 15 absolute points higher than that of the existing methods. This proves that AKE - BML can effectively utilize external knowledge and adjust according to the support set, thereby constructing better event - type prototypes. ### Summary By introducing the Adaptive Knowledge - Enhanced Bayesian Meta - Learning framework, this paper solves the problems of insufficient sample diversity and incomplete coverage of external knowledge in few - shot event detection, and significantly improves the performance and generalization ability of the model.