Abstract:We examine the diverse methodologies used in evaluating graph and network layout algorithms, covering computational evaluations, benchmarks, and user studies, detailing their techniques, outcomes, and trade‐offs. The aim is to guide readers in understanding and applying evaluation methods effectively, providing insights into algorithm performance and suitability for specific tasks. Evaluations—encompassing computational evaluations, benchmarks and user studies—are essential tools for validating the performance and applicability of graph and network layout algorithms (also known as graph drawing). These evaluations not only offer significant insights into an algorithm's performance and capabilities, but also assist the reader in determining if the algorithm is suitable for a specific purpose, such as handling graphs with a high volume of nodes or dense graphs. Unfortunately, there is no standard approach for evaluating layout algorithms. Prior work holds a 'Wild West' of diverse benchmark datasets and data characteristics, as well as varied evaluation metrics and ways to report results. It is often difficult to compare layout algorithms without first implementing them and then running your own evaluation. In this systematic review, we delve into the myriad of methodologies employed to conduct evaluations—the utilized techniques, reported outcomes and the pros and cons of choosing one approach over another. Our examination extends beyond computational evaluations, encompassing user‐centric evaluations, thus presenting a comprehensive understanding of algorithm validation. This systematic review—and its accompanying website—guides readers through evaluation types, the types of results reported, and the available benchmark datasets and their data characteristics. Our objective is to provide a valuable resource for readers to understand and effectively apply various evaluation methods for graph layout algorithms. A free copy of this paper and all supplemental material is available at osf.io, and the categorized papers are accessible on our website at https://visdunneright.github.io/gd‐comp‐eval/.

Layered Graph Drawing for Visualizing Evaluation Structures

Visualizing Hierarchical Social Networks

Experiments and a User Study for Hierarchical Drawings of Graphs

An Experimental Evaluation of Viewpoint‐Based 3D Graph Drawing

DeepDrawing: A Deep Learning Approach to Graph Drawing

Scalable Readability Evaluation for Graph Layouts: 2D Geometric Distributed Algorithms

Graph Based Representation in Design of Multifunctional Structure

Intelligent Graph Layout Using Many Users' Input.

The Analysis and Extraction of Structure from Organizational Charts

Evaluating Graph Layout Algorithms: A Systematic Review of Methods and Best Practices

An Overview + Detail Layout for Visualizing Compound Graphs

A Shape Change Enhancing Hierarchical Layout for the Pairwise Comparison of Directed Acyclic Graphs

Multilevel Visualisation of Topic Dependency Models for Assessment Design and Delivery: A Hypergraph Based Approach

Evaluation of a novel visualization for dynamic social networks

Structure-Aware Simplification for Hypergraph Visualization

Graph Exploration with Embedding-Guided Layouts

Layered Drawing of Undirected Graphs with Generalized Port Constraints

On Low Rank Directed Acyclic Graphs and Causal Structure Learning

A Deep Generative Model for Graph Layout

Extracting Network Structures from Corporate Organization Charts Using Heuristic Image Processing

Sketching Image Gist: Human-Mimetic Hierarchical Scene Graph Generation