LLM-Collaboration on Automatic Science Journalism for the General Audience

Gongyao Jiang,Xinran Shi,Qiong Luo

2024-07-13

Abstract:Science journalism reports current scientific discoveries to non-specialists, aiming to enable public comprehension of the state of the art. However, this task can be challenging as the audience often lacks specific knowledge about the presented research. To address this challenge, we propose a framework that integrates three LLMs mimicking the real-world writing-reading-feedback-revision workflow, with one LLM acting as the journalist, a smaller LLM as the general public reader, and the third LLM as an editor. The journalist's writing is iteratively refined by feedback from the reader and suggestions from the editor. Our experiments demonstrate that by leveraging the collaboration of two 7B and one 1.8B open-source LLMs, we can generate articles that are more accessible than those generated by existing methods, including advanced models such as GPT-4.

Computation and Language

What problem does this paper attempt to address?

### The Problem the Paper Attempts to Solve

The paper aims to address a key challenge in science news reporting: how to transform complex scientific papers into popular science articles that the general public can understand. Specifically, the authors propose a new framework that utilizes three large language models (LLMs) to simulate the real-world processes of writing, reading, feedback, and revision. These three models play the roles of a journalist, an ordinary reader, and an editor, respectively, generating easy-to-understand popular science articles through an iterative process.

### Main Contributions
1. **Proposed a new automatic science news generation framework**: This framework uses three collaboratively working LLMs to generate highly readable popular science articles.
2. **Comprehensive experiments and analysis**: Including comparative experiments on different models and providing detailed analysis and recommendations, demonstrating the effectiveness of LLMs in the task of automatic science news generation.

### Method Overview
- **LLM Journalist**: Responsible for rewriting technical papers into articles aimed at the public.
- **LLM Reader**: Simulates the role of an ordinary reader, reads the article, takes notes, and provides feedback on the article's readability.
- **LLM Editor**: Evaluates the quality of the article based on the reader's feedback and provides revision suggestions.

Through this iterative process, the readability and accuracy of the article are continuously improved, ultimately generating popular science articles suitable for the general public.