Abstract:Script event prediction aims to predict the subsequent event given the context. This requires the capability to infer the correlations between events. Recent works have attempted to improve event correlation reasoning by using pretrained language models and incorporating external knowledge~(e.g., discourse relations). Though promising results have been achieved, some challenges still remain. First, the pretrained language models adopted by current works ignore event-level knowledge, resulting in an inability to capture the correlations between events well. Second, modeling correlations between events with discourse relations is limited because it can only capture explicit correlations between events with discourse markers, and cannot capture many implicit correlations. To this end, we propose a novel generative approach for this task, in which a pretrained language model is fine-tuned with an event-centric pretraining objective and predicts the next event within a generative paradigm. Specifically, we first introduce a novel event-level blank infilling strategy as the learning objective to inject event-level knowledge into the pretrained language model, and then design a likelihood-based contrastive loss for fine-tuning the generative model. Instead of using an additional prediction layer, we perform prediction by using sequence likelihoods generated by the generative model. Our approach models correlations between events in a soft way without any external knowledge. The likelihood-based prediction eliminates the need to use additional networks to make predictions and is somewhat interpretable since it scores each word in the event. Experimental results on the multi-choice narrative cloze~(MCNC) task demonstrate that our approach achieves better results than other state-of-the-art baselines. Our code will be available at <a class="link-external link-https" href="https://github.com/zhufq00/mcnc" rel="external noopener nofollow">this https URL</a>.

Conditional Generation of Temporally-ordered Event Sequences

What Happens Next? Event Prediction Using a Compositional Neural Network Model

Generating Temporally-ordered Event Sequences via Event Optimal Transport.

Temporal Event Knowledge Acquisition via Identifying Narratives

Neural Language Modeling for Contextualized Temporal Graph Generation

Latent Event-Predictive Encodings through Counterfactual Regularization

Narrative Graph: Telling Evolving Stories Based on Event-centric Temporal Knowledge Graph

Sequence-Based Modeling for Temporal Knowledge Graph Link Prediction

Constructing Narrative Event Evolutionary Graph for Script Event Prediction

A Temporal Variational Model for Story Generation

Learning Conditional Generative Models for Temporal Point Processes

A Generative Approach for Script Event Prediction via Contrastive Fine-tuning

Sort story: Sorting jumbled images and captions into stories

Team RUC_AIM3 Technical Report at Activitynet 2020 Task 2: Exploring Sequential Events Detection for Dense Video Captioning

Discourse-Level Event Temporal Ordering with Uncertainty-Guided Graph Completion

The Future is not One-dimensional: Complex Event Schema Induction by Graph Modeling for Event Prediction

A Graph Enhanced BERT Model for Event Prediction

Narrative-of-Thought: Improving Temporal Reasoning of Large Language Models via Recounted Narratives

EtriCA: Event-Triggered Context-Aware Story Generation Augmented by Cross Attention

Dynamic Scene Graph Generation Via Temporal Prior Inference

Learning Sequence Encoders for Temporal Knowledge Graph Completion