Abstract:Visually-situated languages such as charts and plots are omnipresent in real-world documents. These graphical depictions are human-readable and are often analyzed in visually-rich documents to address a variety of questions that necessitate complex reasoning and common-sense responses. Despite the growing number of datasets that aim to answer questions over charts, most only address this task in isolation, without considering the broader context of document-level question answering. Moreover, such datasets lack adequate common-sense reasoning information in their questions. In this work, we introduce a novel task named document-level chart question answering (DCQA). The goal of this task is to conduct document-level question answering, extracting charts or plots in the document via document layout analysis (DLA) first and subsequently performing chart question answering (CQA). The newly developed benchmark dataset comprises 50,010 synthetic documents integrating charts in a wide range of styles (6 styles in contrast to 3 for PlotQA and ChartQA) and includes 699,051 questions that demand a high degree of reasoning ability and common-sense understanding. Besides, we present the development of a potent question-answer generation engine that employs table data, a rich color set, and basic question templates to produce a vast array of reasoning question-answer pairs automatically. Based on DCQA, we devise an OCR-free transformer for document-level chart-oriented understanding, capable of DLA and answering complex reasoning and common-sense questions over charts in an OCR-free manner. Our DCQA dataset is expected to foster research on understanding visualizations in documents, especially for scenarios that require complex reasoning for charts in the visually-rich document. We implement and evaluate a set of baselines, and our proposed method achieves comparable results.

LEAF-QA: Locate, Encode & Attend for Figure Question Answering

RealCQA: Scientific Chart Question Answering as a Test-bed for First-Order Logic

DCQA: Document-Level Chart Question Answering towards Complex Reasoning and Common-Sense Understanding

Synthesize Step-by-Step: Tools, Templates and LLMs as Data Generators for Reasoning-Based Chart VQA

OpenCQA: Open-ended Question Answering with Charts

Charting the Future: Using Chart Question-Answering for Scalable Evaluation of LLM-Driven Data Visualizations

Enhancing Question Answering on Charts Through Effective Pre-training Tasks

mChartQA: A universal benchmark for multimodal Chart Question Answer based on Vision-Language Alignment and Reasoning

Tree-KGQA: An Unsupervised Approach for Question Answering Over Knowledge Graphs

SciGraphQA: A Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific Graphs

ActivityNet-QA: A Dataset for Understanding Complex Web Videos Via Question Answering.

Assertion-based QA with Question-Aware Open Information Extraction

QAEncoder: Towards Aligned Representation Learning in Question Answering System

Localize, Retrieve and Fuse: A Generalized Framework for Free-Form Question Answering over Tables

LogQA: Question Answering in Unstructured Logs

ANetQA: A Large-scale Benchmark for Fine-grained Compositional Reasoning over Untrimmed Videos

MFORT-QA: Multi-hop Few-shot Open Rich Table Question Answering

XAIQA: Explainer-Based Data Augmentation for Extractive Question Answering

Relation-Aware Fine-Grained Reasoning Network for Textbook Question Answering

Advancing Multimodal Large Language Models in Chart Question Answering with Visualization-Referenced Instruction Tuning