Abstract:Large language models appear quite creative, often performing on par with the average human on creative tasks. However, research on LLM creativity has focused solely on \textit{products}, with little attention on the creative \textit{process}. Process analyses of human creativity often require hand-coded categories or exploit response times, which do not apply to LLMs. We provide an automated method to characterise how humans and LLMs explore semantic spaces on the Alternate Uses Task, and contrast with behaviour in a Verbal Fluency Task. We use sentence embeddings to identify response categories and compute semantic similarities, which we use to generate jump profiles. Our results corroborate earlier work in humans reporting both persistent (deep search in few semantic spaces) and flexible (broad search across multiple semantic spaces) pathways to creativity, where both pathways lead to similar creativity scores. LLMs were found to be biased towards either persistent or flexible paths, that varied across tasks. Though LLMs as a population match human profiles, their relationship with creativity is different, where the more flexible models score higher on creativity. Our dataset and scripts are available on \href{<a class="link-external link-https" href="https://github.com/surabhisnath/Creative_Process" rel="external noopener nofollow">this https URL</a>}{GitHub}.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to explore and characterize the creative processes of humans and large language models (LLMs) in creative tasks. Specifically, the research aims to: 1. **Fill the research gap**: Previous research on LLM creativity has mainly focused on the product level, that is, evaluating whether the content generated by the model is original and useful, while ignoring the study of the creative process. This paper focuses on the creative process and attempts to understand "how creativity is generated". 2. **Compare the creative processes of humans and LLMs**: By analyzing how humans and LLMs explore the semantic space when performing the Alternate Uses Task (AUT) and the Verbal Fluency Task (VFT), compare the differences and similarities between the two. 3. **Develop an automated method**: Propose a fully automated, data - driven method to identify jump signals in response sequences, thereby quantifying and characterizing the persistent and flexible paths in the creative process. ### Main research questions - **Characterization of the creative process**: How can an automated method be used to characterize the exploration behaviors of humans and LLMs in creative tasks? - **Path differences**: Are there different paths (such as persistence or flexibility) in the creative processes of humans and LLMs? What impact do these paths have on the creativity score? - **Model performance**: Is the creative performance of LLMs in different tasks similar to that of humans? Are there significant differences in their flexibility and persistence paths? ### Method overview The researchers used sentence embeddings to identify response categories and calculate semantic similarity, and then generated jump profiles. Through this method, they were able to analyze the response patterns of humans and LLMs in AUT and VFT tasks, identify persistent and flexible paths, and correlate these paths with creativity scores. ### Key findings - **Jump profiles of humans and LLMs**: Humans and LLMs show similar jump profiles in the AUT task, but in the VFT task, LLMs are more inclined to the persistent path. - **Flexibility and creativity**: For humans, the flexible path is not necessarily more creative than the persistent path; for LLMs, the flexible path is associated with a higher creativity score. - **Differences between models**: Different LLMs show different preferences in the AUT task, some are biased towards flexibility, and some are biased towards persistence, and this preference is not consistent across different tasks. Through these studies, the authors hope to provide new perspectives and tools for future cross - research of artificial intelligence and cognitive science, and further understand the creative processes of humans and machines.

Characterising the Creative Process in Humans and Large Language Models

Divergent Creativity in Humans and Large Language Models

Large Language Models show both individual and collective creativity comparable to humans

On the Creativity of Large Language Models

Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking

The Language of Creativity: Evidence from Humans and Large Language Models

Assessing and Understanding Creativity in Large Language Models

Evaluating Creative Short Story Generation in Humans and Large Language Models

Evaluating Large Language Model Creativity from a Literary Perspective

Steering Large Language Models to Evaluate and Amplify Creativity

Art or Artifice? Large Language Models and the False Promise of Creativity

Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writers

Homogenization Effects of Large Language Models on Human Creative Ideation

Automatic Scoring of Metaphor Creativity with Large Language Models

LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play

Probing the Creativity of Large Language Models: Can models produce divergent semantic association?

Enhancing Creativity in Large Language Models through Associative Thinking Strategies

Small Language Models can Outperform Humans in Short Creative Writing: A Study Comparing SLMs with Humans and LLMs

Large Language Models are Fixated by Red Herrings: Exploring Creative Problem Solving and Einstellung Effect using the Only Connect Wall Dataset

Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation

Creative Problem Solving in Large Language and Vision Models -- What Would it Take?