Abstract:Charts are a visualization tool used in scientific documents to facilitate easy comprehension of complex relationships underlying data and experiments. Researchers use various chart types to convey scientific information, so the problem of data extraction and subsequent chart understanding becomes very challenging. Many studies have been taken up in the literature to address the problem of chart mining, whose motivation is to facilitate the editing of existing charts, carry out extrapolative studies, and provide a deeper understanding of the underlying data. The first step towards chart understanding is chart classification, for which traditional ML and CNN-based deep learning models have been used in the literature. In this paper, we propose Swin-Chart, a Swin transformer-based deep learning approach for chart classification, which generalizes well across multiple datasets with a wide range of chart categories. Swin-Chart comprises a pre-trained Swin Transformer, a finetuning component, and a weight averaging component. The proposed approach is tested on a five-chart image benchmark dataset. We observed that the Swin-Chart model outperformers existing state-of-the-art models on all the datasets. Furthermore, we also provide an ablation study of the Swin-Chart model with all five datasets to understand the importance of various sub-parts such as the back-bone Swin transformer model, the value of several best weights selected for the weight averaging component, and the presence of the weight averaging component itself. The Swin-Chart model also received first position in the chart classification task on the latest dataset (private) in the CHART Infographics competition at ICDAR 2023 - chartinfo.github.io.

DeepChart: Combining Deep Convolutional Networks and Deep Belief Networks in Chart Classification.

Chart Classification by Combining Deep Convolutional Networks and Deep Belief Networks

Combining Convolutional and Recurrent Neural Networks for Classification of Focal Liver Lesions in Multi-phase CT Images.

ChartEye: A Deep Learning Framework for Chart Information Extraction

Swin-chart: An efficient approach for chart classification

A Survey and Approach to Chart Classification

Data Extraction from Charts via Single Deep Neural Network

C2F-CHART: A Curriculum Learning Approach to Chart Classification

ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules

Advancing Chart Question Answering with Robust Chart Component Recognition

An Intelligent Approach to Automatically Discovering Visual Insights

ChartLine: Automatic Detection and Tracing of Curves in Scientific Line Charts Using Spatial-Sequence Feature Pyramid Network

Chartem: Reviving Chart Images with Data Embedding

ChartBench: A Benchmark for Complex Visual Reasoning in Charts

EvoChart: A Benchmark and a Self-Training Approach Towards Real-World Chart Understanding

Learning Deep Classifiers With Deep Features

Enhanced Chart Understanding in Vision and Language Task via Cross-modal Pre-training on Plot Table Pairs

Multi-label classification of line chart images using convolutional neural networks

Classification of Electrocardiogram Signals with Deep Belief Networks

ChartNavigator: An Interactive Pattern Identification and Annotation Framework for Charts

TinyChart: Efficient Chart Understanding with Visual Token Merging and Program-of-Thoughts Learning