Writer-Defined AI Personas for On-Demand Feedback Generation

Karim Benharrak,Tim Zindulka,Florian Lehmann,Hendrik Heuer,Daniel Buschek

DOI: https://doi.org/10.1145/3613904.3642406

2024-02-20

Abstract:Compelling writing is tailored to its audience. This is challenging, as writers may struggle to empathize with readers, get feedback in time, or gain access to the target group. We propose a concept that generates on-demand feedback, based on writer-defined AI personas of any target audience. We explore this concept with a prototype (using GPT-3.5) in two user studies (N=5 and N=11): Writers appreciated the concept and strategically used personas for getting different perspectives. The feedback was seen as helpful and inspired revisions of text and personas, although it was often verbose and unspecific. We discuss the impact of on-demand feedback, the limited representativity of contemporary AI systems, and further ideas for defining AI personas. This work contributes to the vision of supporting writers with AI by expanding the socio-technical perspective in AI tool design: To empower creators, we also need to keep in mind their relationship to an audience.

Human-Computer Interaction,Computation and Language

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to solve the problem that the author has difficulty obtaining feedback from the target audience during the writing process. Specifically, the authors propose a new concept - generating on - demand feedback based on author - defined AI personas. This method can help authors better understand and cater to the needs of their target audience, thereby improving the quality of writing. The following are the main focuses of this research:

1. **Disconnection between writing and the audience**: Existing writing tools usually provide only limited support to help authors understand their target audience. This causes authors to have difficulty writing from the perspective of the reader during writing, which in turn affects the readability and relevance of the text.

2. **Challenges in obtaining immediate feedback**: It is very difficult for authors to obtain real - time feedback from the target audience in a timely manner during the writing process, especially when time and resources are limited. Therefore, authors are often unable to make real - time adjustments and improvements based on the opinions of the readers.

3. **Introducing the concept of AI personas**: In order to overcome the above challenges, the authors propose a new writing - aid tool - Impressona. This tool allows authors to create AI personas representing their target audience and generate immediate feedback through these personas. This feedback is generated by large - language models (such as GPT - 3.5) based on the provided persona information.

4. **Evaluating the effectiveness of AI personas**: The authors evaluate the effectiveness of this concept through two user studies (N = 5 and N = 11). The research shows that although the generated feedback is sometimes too long - winded and not specific enough, it is generally considered helpful by the participants and can inspire further modifications to the text and personas.

In conclusion, the core problem of this paper is to help authors obtain more effective immediate feedback from the target audience during the writing process by introducing the concept of AI personas, thereby improving the quality and relevance of writing.