Towards Automated Infographic Design: Deep Learning-based Auto-Extraction of Extensible Timeline

Chen Zhu-Tian,Yun Wang,Qianwen Wang,Yong Wang,Huamin Qu

DOI: https://doi.org/10.1109/TVCG.2019.2934810

2023-10-07

Abstract:Designers need to consider not only perceptual effectiveness but also visual styles when creating an infographic. This process can be difficult and time consuming for professional designers, not to mention non-expert users, leading to the demand for automated infographics design. As a first step, we focus on timeline infographics, which have been widely used for centuries. We contribute an end-to-end approach that automatically extracts an extensible timeline template from a bitmap image. Our approach adopts a deconstruction and reconstruction paradigm. At the deconstruction stage, we propose a multi-task deep neural network that simultaneously parses two kinds of information from a bitmap timeline: 1) the global information, i.e., the representation, scale, layout, and orientation of the timeline, and 2) the local information, i.e., the location, category, and pixels of each visual element on the timeline. At the reconstruction stage, we propose a pipeline with three techniques, i.e., Non-Maximum Merging, Redundancy Recover, and DL GrabCut, to extract an extensible template from the infographic, by utilizing the deconstruction results. To evaluate the effectiveness of our approach, we synthesize a timeline dataset (4296 images) and collect a real-world timeline dataset (393 images) from the Internet. We first report quantitative evaluation results of our approach over the two datasets. Then, we present examples of automatically extracted templates and timelines automatically generated based on these templates to qualitatively demonstrate the performance. The results confirm that our approach can effectively extract extensible templates from real-world timeline infographics.

Human-Computer Interaction,Graphics

What problem does this paper attempt to address?

### Problems Addressed by the Paper

This paper aims to solve a key problem in automatic infographic design: automatically extracting scalable timeline templates from bitmap images. Specifically, the authors focus on how to convert existing timeline infographics (usually in bitmap format) into editable and scalable templates, allowing users to easily update data and generate new timelines.

### Background and Challenges

1. **Complexity of Infographic Design**:
- Designing infographics requires consideration of perceptual effects, visual style, aesthetics, and interactivity.
- This process is already very time-consuming and difficult for professional designers, and even more so for non-professionals.

2. **Limitations of Existing Tools**:
- Existing design tools, while capable of automating certain processes (such as visual encoding), still require users to manually initialize most of the design work (such as drawing graphic elements).
- This makes the entire process still difficult and time-consuming, especially for non-professionals.

3. **Diversity of Templates**:
- Commercial software (such as Microsoft PowerPoint and Adobe Illustrator) offers limited types of templates, lacking diversity.
- Many infographics with diverse styles can only be accessed in bitmap format, and if users want to mimic these styles, they must manually create their own infographics, which is both difficult and tedious.

### Solution

The authors propose an end-to-end approach to address the above issues through the following two phases:

1. **Deconstruction Phase**:
- Using a multi-task deep neural network (DNN) to simultaneously parse both global and local information of the timeline image.
- Global information includes the representation, scale, layout, and orientation of the timeline.
- Local information includes the position, category, and pixels of each visual element.

2. **Reconstruction Phase**:
- Utilizing the results from the deconstruction phase, three techniques (non-maximum merging, redundancy recovery, and DL GrabCut) are used to extract scalable templates from the infographic.
- The extracted templates can be used to generate new timelines with updated data.

### Evaluation and Results

- **Datasets**:
- Synthetic dataset (4296 annotated images).
- Real-world dataset (393 images collected from the internet).

- **Evaluation Methods**:
- Quantitative Evaluation: Reported quantitative evaluation results for both phases on the two datasets.
- Qualitative Demonstration: Showcased the automatically extracted templates and the timelines generated based on these templates to demonstrate performance.

### Main Contributions

- **Automatic Extraction of Scalable Templates**: Proposed a method to automatically extract scalable templates from bitmap timeline infographics.
- **Multi-task DNN**: Designed a multi-task deep neural network capable of simultaneously parsing both global and local information of timeline images.
- **Automatic Reconstruction Pipeline**: Proposed a pipeline that uses deconstruction results to automatically reconstruct scalable templates.

### Conclusion

This study successfully addresses the challenge of automatically extracting scalable templates from bitmap timeline infographics, providing an effective solution for automatic infographic design. Experimental validation shows that this method performs well on real-world datasets, effectively extracting and generating new timelines.