Plan-And-Write: Towards Better Automatic Storytelling

Lili Yao,Nanyun Peng,Ralph Weischedel,Kevin Knight,Dongyan Zhao,Rui Yan

2019-02-19

Abstract:Automatic storytelling is challenging since it requires generating long, coherent natural language to describes a sensible sequence of events. Despite considerable efforts on automatic story generation in the past, prior work either is restricted in plot planning, or can only generate stories in a narrow domain. In this paper, we explore open-domain story generation that writes stories given a title (topic) as input. We propose a plan-and-write hierarchical generation framework that first plans a storyline, and then generates a story based on the storyline. We compare two planning strategies. The dynamic schema interweaves story planning and its surface realization in text, while the static schema plans out the entire storyline before generating stories. Experiments show that with explicit storyline planning, the generated stories are more diverse, coherent, and on topic than those generated without creating a full plan, according to both automatic and human evaluations.

Computation and Language

What problem does this paper attempt to address?

### The Problem the Paper Attempts to Solve

This paper aims to address the key challenges in automatic story generation, specifically how to generate coherent and meaningful stories. The paper focuses on the following aspects:

1. **Diversity**: The generated stories should be diverse and avoid repetition.
2. **Coherence**: The generated stories should be logically coherent with a reasonable sequence of events.
3. **Theme Relevance**: The generated stories should be closely related to the given theme or title.

### Background

Automatic story generation is a complex task because it requires generating long and coherent natural language to describe a series of logically consistent events. Although past research has made some progress in automatic story generation, most methods are either limited in plot planning or can only generate stories in specific domains. These methods usually rely on a large amount of manual annotation and mostly remain at the level of abstract story representation without achieving surface realization in natural language.

### Solution

To overcome the above problems, the paper proposes a hierarchical generation framework called "Plan-And-Write," which consists of two steps:

1. **Plot Planning**: Generate a story outline based on the given title or theme.
2. **Surface Realization**: Generate the specific natural language story based on the plot outline.

### Main Contributions

1. **Framework Design**: Proposes the "Plan-And-Write" framework to improve the diversity and coherence of generated stories through plot planning.
2. **Strategy Comparison**: Explores and compares two plot planning strategies—dynamic planning and static planning.
3. **Evaluation Metrics**: Develops new evaluation metrics to measure the diversity of generated stories and conducts novel analyses to assess the importance of different story aspects to human evaluation.
4. **Experimental Results**: Experiments show that the proposed "Plan-And-Write" model generates stories with greater diversity, coherence, and theme relevance compared to methods without planning.

### Methodology

- **Dynamic Planning**: Adjusts the plot step by step during the writing process, generating the next plot word and sentence at each step.
- **Static Planning**: Generates a complete plot outline before writing and then generates the story based on the outline.

### Experimental Setup

- **Dataset**: Uses the ROCStories corpus, containing 98,162 short stories as training data and 1,817 stories for development and testing.
- **Baseline Methods**: Compares with representative baseline methods that do not include a planning module, including Incremental Sentence Generation (Inc-S2S) and Conditional Language Model (Cond-LM).
- **Hyperparameter Optimization**: Performs the same hyperparameter optimization for all models, using Stochastic Gradient Descent (SGD) for training.

### Evaluation Metrics

- **Objective Metrics**: Designs new automatic evaluation metrics, including cross-story and within-story repetition rates, to quantify the diversity of generated stories.
- **Subjective Metrics**: Evaluates through human assessment in terms of theme relevance, coherence, interestingness, and overall user preference.

### Results Discussion

- **Objective Evaluation**: Experimental results show that the "Plan-And-Write" framework significantly reduces the repetition rate of generated stories and increases diversity.
- **Subjective Evaluation**: Human evaluation results indicate that both dynamic planning and static planning significantly outperform the corresponding baseline methods, with static planning performing the best.

### Conclusion

By introducing the "Plan-And-Write" framework, this paper effectively addresses the issues of diversity and coherence in automatic story generation. The experimental results validate the effectiveness of this approach, providing new directions for future research.