Lu Ying,Yun Wang,Haotian Li,Shuguang Dou,Haidong Zhang,Xinyang Jiang,Huamin Qu,Yingcai Wu
Abstract:Data charts are prevalent across various fields due to their efficacy in conveying complex data relationships. However, static charts may sometimes struggle to engage readers and efficiently present intricate information, potentially resulting in limited understanding. We introduce "Live Charts," a new format of presentation that decomposes complex information within a chart and explains the information pieces sequentially through rich animations and accompanying audio narration. We propose an automated approach to revive static charts into Live Charts. Our method integrates GNN-based techniques to analyze the chart components and extract data from charts. Then we adopt large natural language models to generate appropriate animated visuals along with a voice-over to produce Live Charts from static ones. We conducted a thorough evaluation of our approach, which involved the model performance, use cases, a crowd-sourced user study, and expert interviews. The results demonstrate Live Charts offer a multi-sensory experience where readers can follow the information and understand the data insights better. We analyze the benefits and drawbacks of Live Charts over static charts as a new information consumption experience.
What problem does this paper attempt to address?
### Problems the Paper Aims to Solve
This paper aims to address the limitations of static charts in conveying complex data relationships. Specifically, static charts, due to their unchanging nature, may struggle to capture the reader's attention and fail to efficiently present complex, multi-layered information, which can lead to limitations in understanding. Additionally, static charts have limitations in terms of information-carrying capacity; too many visual components or encodings may cause visual overload for users.
To solve these issues, the authors introduce a new chart presentation format—"Live Charts." This format improves user understanding and engagement by breaking down complex information into multiple parts and explaining these parts sequentially through rich animations and accompanying audio narration. The authors propose an automated method to convert static charts into live charts, utilizing Graph Neural Network (GNN) technology to analyze chart components and extract data, then employing large natural language models to generate corresponding animations and voice narrations, ultimately producing live charts.
### Main Contributions
1. **Automated Pipeline**: An automated pipeline is proposed to convert static charts into live charts. This method includes:
- Using a dual-stream Graph Neural Network (GNN) to recover data and visual encodings from SVG charts.
- Automatically generating narrations and animations from the recovered data to create live charts.
2. **Multisensory Experience**: Live charts provide a multisensory experience through synchronized animations and audio narrations, helping users better understand and follow the information in the charts.
3. **Comprehensive Evaluation**: The effectiveness of the method is validated through various evaluation methods, including model performance evaluation, real-world application cases, crowdsourced user studies, and expert interviews.
### Related Work
1. **Visualization Understanding**: Research on how to automatically understand the content of existing charts, including chart types, data, and visual encodings. Existing methods are mainly divided into raster image-based and vector image-based approaches. Vector images, due to their rich structural information, are more suitable for automatic chart understanding.
2. **Visualization Enhancement**: Research on how to enhance the expressiveness of charts by adding question answering, annotations, and titles, helping users understand charts more quickly and effectively.
3. **Data Video Generation**: Research on how to enhance data visualization by creating data videos, including integrating animations and developing systems to automatically generate animations. Existing methods mainly focus on specific input formats, whereas the method in this paper can handle various SVG format charts.
### Method for Generating Live Charts
1. **Chart Understanding**: Using GNN technology to identify data and visual encodings from the input SVG file.
2. **Generating Narration**: Utilizing large language models (such as GPT-3) to generate high-quality audio narrations based on the chart data.
3. **Designing Animations**: Designing corresponding animations based on the generated narrations and synchronizing them with the audio to create live charts.
Through this method, the authors hope to improve user understanding and engagement with complex data, making charts a more effective communication tool.