Abstract:Recent studies on computer vision mainly focus on natural images that express real-world scenes. They achieve outstanding performance on diverse tasks such as visual question answering. Diagram is a special form of visual expression that frequently appears in the education field and is of great significance for learners to understand multimodal knowledge. Current research on diagrams preliminarily focuses on natural disciplines such as Biology and Geography, whose expressions are still similar to natural images. Another type of diagrams such as from Computer Science is composed of graphics containing complex topologies and relations, and research on this type of diagrams is still blank. The main challenges of graphic diagrams understanding are the rarity of data and the confusion of semantics, which are mainly reflected in the diversity of expressions. In this paper, we construct a novel dataset of graphic diagrams named Computer Science Diagrams (CSDia). It contains more than 1,200 diagrams and exhaustive annotations of objects and relations. Considering the visual noises caused by the various expressions in diagrams, we introduce the topology of diagrams to parse topological structure. After that, we propose Diagram Parsing Net (DPN) to represent the diagram from three branches: topology, visual feature, and text, and apply the model to the diagram classification task to evaluate the ability of diagrams understanding. The results show the effectiveness of the proposed DPN on diagrams understanding.

DFDVis: A Visual Analytics System for Understanding the Semantics of Data Flow Diagram.

Analyzing Data Flow Diagrams by Combination of Formal Methods and Visualization Techniques.

Design of Data Visualization System for Statistical Applications

Dart-Dataflow: Towards Communicating Data Semantics in Sensor Grid.

ZFDV: Zoom Financial Data Visualization in the Era of Big Data

A Model-Driven Visualization System Based On Dvdl

DESIGN AND IMPLEMENTATION OF A GENERAL INFORMATION VISUALIZATION MODEL

Semantic Flow Graph : A Framework to Explore 3 D Flow Fields

VisDAMS:A Visual Scientific Data Management System

Model-Driven Visualization Generation System

How Dataflow Diagrams Impact Software Security Analysis: an Empirical Experiment

An Extensible Framework for Architecture-Based Data Flow Analysis for Information Security

Semantic Flow Graph: A Framework for Discovering Object Relationships in Flow Fields

Formalization of the data flow diagram rules for consistency check

Fuzzy-based Description and Interactive Extraction of Features in 3D Flows

Visualizing Relationships for Real World Applications

RL-CSDia: Representation Learning of Computer Science Diagrams

Discovering Fuzzy Functional Dependencies as Semantic Knowledge in Large Databases

Semantics and Verification of Data Flow in UML Statecharts

DGSVis: Visual Analysis of Hierarchical Snapshots in Dynamic Graph

Big data dimensional analysis