Abstract:Script event prediction aims to predict the subsequent event given the context. This requires the capability to infer the correlations between events. Recent works have attempted to improve event correlation reasoning by using pretrained language models and incorporating external knowledge~(e.g., discourse relations). Though promising results have been achieved, some challenges still remain. First, the pretrained language models adopted by current works ignore event-level knowledge, resulting in an inability to capture the correlations between events well. Second, modeling correlations between events with discourse relations is limited because it can only capture explicit correlations between events with discourse markers, and cannot capture many implicit correlations. To this end, we propose a novel generative approach for this task, in which a pretrained language model is fine-tuned with an event-centric pretraining objective and predicts the next event within a generative paradigm. Specifically, we first introduce a novel event-level blank infilling strategy as the learning objective to inject event-level knowledge into the pretrained language model, and then design a likelihood-based contrastive loss for fine-tuning the generative model. Instead of using an additional prediction layer, we perform prediction by using sequence likelihoods generated by the generative model. Our approach models correlations between events in a soft way without any external knowledge. The likelihood-based prediction eliminates the need to use additional networks to make predictions and is somewhat interpretable since it scores each word in the event. Experimental results on the multi-choice narrative cloze~(MCNC) task demonstrate that our approach achieves better results than other state-of-the-art baselines. Our code will be available at <a class="link-external link-https" href="https://github.com/zhufq00/mcnc" rel="external noopener nofollow">this https URL</a>.

What Comes Next and Why? A Staged Encoder-Decoder Architecture for Script Event Prediction

What happens next? Combining enhanced multilevel script learning and dual fusion strategies for script event prediction

Integrating Deep Event-Level and Script-Level Information for Script Event Prediction

What Would Happen Next? Predicting Consequences from An Event Causality Graph

Constructing Narrative Event Evolutionary Graph for Script Event Prediction

Multi-level Connection Enhanced Representation Learning for Script Event Prediction

A Generative Approach for Script Event Prediction via Contrastive Fine-tuning

Going Beyond Multi-Task Dense Prediction with Synergy Embedding Models

What Happens Next? Event Prediction Using a Compositional Neural Network Model

Prompt2Gaussia: Uncertain Prompt-learning for Script Event Prediction

Embedding and Predicting the Event at Early Stage

VEDIT: Latent Prediction Architecture For Procedural Video Representation Learning

ONSEP: A Novel Online Neural-Symbolic Framework for Event Prediction Based on Large Language Model

Seer: Language Instructed Video Prediction with Latent Diffusion Models.

Two-stage scheduling of integrated energy systems based on a two-step DCGAN-based scenario prediction approach

What Happens Next? Future Subevent Prediction Using Contextual Hierarchical LSTM.

Accurate and Efficient Event-based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Network

Event Representations With Tensor-Based Compositions

Object-Centric Cross-Modal Knowledge Reasoning for Future Event Prediction in Videos

Cross Scene Prediction via Modeling Dynamic Correlation using Latent Space Shared Auto-Encoders

A neural model for joint event detection and prediction