Abstract:The statistical study of human memory requires large-scale experiments, involving many stimuli conditions and test subjects. While this approach has proven to be quite fruitful for meaningless material such as random lists of words, naturalistic stimuli, like narratives, have until now resisted such a large-scale study, due to the quantity of manual labor required to design and analyze such experiments. In this work, we develop a pipeline that uses large language models (LLMs) both to design naturalistic narrative stimuli for large-scale recall and recognition memory experiments, as well as to analyze the results. We performed online memory experiments with a large number of participants and collected recognition and recall data for narratives of different sizes. We found that both recall and recognition performance scale linearly with narrative length; however, for longer narratives people tend to summarize the content rather than recalling precise details. To investigate the role of narrative comprehension in memory, we repeated these experiments using scrambled versions of the narratives. Although recall performance declined significantly, recognition remained largely unaffected. Recalls in this condition seem to follow the original narrative order rather than the actual scrambled presentation, pointing to a contextual reconstruction of the story in memory. Finally, using LLM text embeddings, we construct a simple measure for each clause based on semantic similarity to the whole narrative, that shows a strong correlation with recall probability. Overall, our work demonstrates the power of LLMs in accessing new regimes in the study of human memory, as well as suggesting novel psychologically informed benchmarks for LLM performance.

Imagined versus Remembered Stories: Quantifying Differences in Narrative Flow

Quantifying the narrative flow of imagined versus autobiographical stories

Understanding Narrative Linearity for Telling Expressive Time-Oriented Stories

Large-scale study of human memory for meaningful narratives

Narrative Variations in a Virtual Storyteller

Visual Narrative Flow: Exploring Factors Shaping Data Visualization Story Reading Experiences

Testing the Quantitative Spacetime Hypothesis using Artificial Narrative Comprehension (I) : Bootstrapping Meaning from Episodic Narrative viewed as a Feature Landscape

Fine-Grained Modeling of Narrative Context: A Coherence Perspective via Retrospective Questions

Random Tree Model of Meaningful Memory

Stories, Statistics, and Memory

Learner agency and potential for creativity in writing task design and Chinese EFL students’ flow experiences in writing: A contextual perspective

Bigram semantic distance as an index of continuous semantic flow in natural language: Theory, tools, and applications.

Exploring the affordances of text and picture stories

Imagining is not seeing: lower insight-driven memory reconfiguration when imagining the link between separate events

Information flow across the cortical timescale hierarchy during narrative construction

Memory and comprehension of narrative versus expository texts: A meta-analysis

Taking the unreal seriously: enriching cognitive science with the notion of fictionality

Remembering the past and imagining the future: Differences in event specificity of spontaneously generated thought

How Concrete Do We Get Telling Stories?

On Narrative Information and the Distillation of Stories

Quantitative text feature analysis of autobiographical interview data: prediction of episodic details, semantic details and temporal discounting