Abstract:We examine the diverse methodologies used in evaluating graph and network layout algorithms, covering computational evaluations, benchmarks, and user studies, detailing their techniques, outcomes, and trade‐offs. The aim is to guide readers in understanding and applying evaluation methods effectively, providing insights into algorithm performance and suitability for specific tasks. Evaluations—encompassing computational evaluations, benchmarks and user studies—are essential tools for validating the performance and applicability of graph and network layout algorithms (also known as graph drawing). These evaluations not only offer significant insights into an algorithm's performance and capabilities, but also assist the reader in determining if the algorithm is suitable for a specific purpose, such as handling graphs with a high volume of nodes or dense graphs. Unfortunately, there is no standard approach for evaluating layout algorithms. Prior work holds a 'Wild West' of diverse benchmark datasets and data characteristics, as well as varied evaluation metrics and ways to report results. It is often difficult to compare layout algorithms without first implementing them and then running your own evaluation. In this systematic review, we delve into the myriad of methodologies employed to conduct evaluations—the utilized techniques, reported outcomes and the pros and cons of choosing one approach over another. Our examination extends beyond computational evaluations, encompassing user‐centric evaluations, thus presenting a comprehensive understanding of algorithm validation. This systematic review—and its accompanying website—guides readers through evaluation types, the types of results reported, and the available benchmark datasets and their data characteristics. Our objective is to provide a valuable resource for readers to understand and effectively apply various evaluation methods for graph layout algorithms. A free copy of this paper and all supplemental material is available at osf.io, and the categorized papers are accessible on our website at https://visdunneright.github.io/gd‐comp‐eval/.

Evaluating Graph Layout Algorithms: A Systematic Review of Methods and Best Practices

Evaluation of Graph Layout Methods Based on Visual Perception.

Overall quality evaluation of graph layouts based on regression analysis

What Would a Graph Look Like in this Layout? A Machine Learning Approach to Large Graph Visualization

Scalable Readability Evaluation for Graph Layouts: 2D Geometric Distributed Algorithms

Evaluation of Graph Sampling: A Visualization Perspective

Evaluation Of Complex Network Layout Based On User Cognition

Evaluation of Graph Sampling

Performance Comparisons Between Force-Directed Algorithms on Structured Data Analysis

Evaluation and Analysis of Distributed Graph-Parallel Processing Frameworks

A Scalable Method for Readable Tree Layouts

Intelligent Graph Layout Using Many Users' Input.

Quality Metrics and Reordering Strategies for Revealing Patterns in BioFabric Visualizations

State and trend of optimization algorithms for layout design

A Graph Model and a Layout Algorithm for Knitting Patterns

Theory and Methodology an Algorithm of Global Optimization for Solving Layout Problems

SoK: The Faults in our Graph Benchmarks

A systematic review on the practice of evaluating visualization

An Experimental Evaluation of Viewpoint‐Based 3D Graph Drawing

Distributed Graph Layout for Scalable Small-world Network Analysis

Striking the Right Balance: Systematic Assessment of Evaluation Method Distribution Across Contribution Types