Narrative Player: Reviving Data Narratives with Visuals

Zekai Shao,Leixian Shen,Haotian Li,Yi Shan,Huamin Qu,Yun Wang,Siming Chen

2024-10-04

Abstract:Data-rich documents are commonly found across various fields such as business, finance, and science. However, a general limitation of these documents for reading is their reliance on text to convey data and facts. Visual representation of text aids in providing a satisfactory reading experience in comprehension and engagement. However, existing work emphasizes presenting the insights of local text context, rather than fully conveying data stories within the whole paragraphs and engaging readers. To provide readers with satisfactory data stories, this paper presents Narrative Player, a novel method that automatically revives data narratives with consistent and contextualized visuals. Specifically, it accepts a paragraph and corresponding data table as input and leverages LLMs to characterize the clauses and extract contextualized data facts. Subsequently, the facts are transformed into a coherent visualization sequence with a carefully designed optimization-based approach. Animations are also assigned between adjacent visualizations to enable seamless transitions. Finally, the visualization sequence, transition animations, and audio narration generated by text-to-speech technologies are rendered into a data video. The evaluation results showed that the automatic-generated data videos were well-received by participants and experts for enhancing reading.

Human-Computer Interaction

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **How to enhance the reading experience and user engagement of data - rich documents through automated vision and animation generation techniques**. Specifically, existing data - rich documents (such as reports in the commercial, financial, and scientific fields) usually rely on plain - text forms to convey data and facts. This method has limitations in data understanding and is likely to lead to problems such as difficulty in interpretation, information fragmentation, and low user engagement.

To solve these problems, the author proposes a new method named **Narrative Player**, which aims to enhance the reading experience by automatically transforming data narratives into consistent and context - related visual sequences. Specific goals include:

1. **Understand data narratives and extract text intentions**: Analyze the input narrative text, understand its explicit and implicit text intentions, and extract key data facts from it.
2. **Handle ambiguous expressions in context**: By analyzing the context in which sentences appear, determine the most likely interpretation to ensure the coherence of the narrative even in the face of the ambiguity of natural language.
3. **Align text and visual intentions**: Ensure the seamless combination between the narrative and visual elements, and use similar visual elements or themes to reinforce the key ideas or concepts presented in the text.
4. **Ensure visual consistency and contextualization**: Maintain consistency in style, color, and layout to match the theme and tone of the narrative, while balancing visual diversity and the user's cognitive load.
5. **Achieve multi - channel perception through transitions, audio, and subtitles**: Utilize dynamic transition effects and audio narration to enhance the user experience, so that the finally generated video can accurately convey the original text information.

Through these means, Narrative Player aims to provide a brand - new way, making data - rich documents not only easy to understand but also more vivid and interesting, thereby enhancing the user's reading experience and engagement.