Towards Scalable and Practical Batch-Dynamic Connectivity

Quinten De Man,Laxman Dhulipala,Adam Karczmarz,Jakub Łącki,Julian Shun,Zhongqi Wang

2024-11-19

Abstract:We study the problem of dynamically maintaining the connected components of an undirected graph subject to edge insertions and deletions. We give the first parallel algorithm for the problem which is work-efficient, supports batches of updates, runs in polylogarithmic depth, and uses only linear total space. The existing algorithms for the problem either use super-linear space, do not come with strong theoretical bounds, or are not parallel. On the empirical side, we provide the first implementation of the cluster forest algorithm, the first linear-space and poly-logarithmic update time algorithm for dynamic connectivity. Experimentally, we find that our algorithm uses up to 19.7x less space and is up to 6.2x faster than the level-set algorithm of HDT, arguably the most widely-implemented dynamic connectivity algorithm with strong theoretical guarantees.

Data Structures and Algorithms,Databases,Distributed, Parallel, and Cluster Computing

What problem does this paper attempt to address?

This paper attempts to solve the problem of dynamic graph connectivity, especially how to efficiently maintain the connected components of an undirected graph under edge insertion and deletion operations. Specifically, the paper focuses on the following aspects:

1. **High - efficiency of work**: The algorithm needs to maintain a low work complexity (i.e., the total number of operations) when processing batch updates.
2. **Support for batch updates**: It can handle the insertion or deletion operations of multiple edges at one time.
3. **Parallel computing**: It can run efficiently in a multi - thread or multi - processor environment and has a low time - depth (i.e., the length of the longest dependency chain).
4. **Linear space usage**: The algorithm should only use a linear total space to ensure its practicality on large - scale graphs.

### Research Background and Existing Challenges

The existing dynamic connectivity algorithms have the following deficiencies:
- **High space cost**: For example, the HDT algorithm requires up to 70 times the space of a simple static representation when processing large - scale graphs.
- **Lack of theoretical guarantees**: Some algorithms are practical but lack strict theoretical performance guarantees.
- **Non - parallelizable**: Many efficient dynamic connectivity algorithms are difficult to parallelize, which limits their application on modern multi - core processors.

### Main Contributions of the Paper

1. **Proposing a new parallel algorithm**: For the first time, the paper proposes a parallel dynamic connectivity algorithm. This algorithm is not only efficient in work, but also supports batch updates and can run efficiently in a multi - thread environment while maintaining linear total space usage.
2. **Introducing blocking invariants**: In order to ensure that the diameter of the clustered graph is small (at most 2), so that graph search can be efficiently parallelized, the paper introduces the concept of "blocking edges" and "blocking invariants".
3. **Experimental verification**: Through experimental comparison, the paper shows the superiority of the new algorithm in practical applications, especially significantly better than the existing HDT algorithm in terms of space efficiency and running speed.

### Specific Technical Details

The paper achieves these goals in the following ways:

- **Blocking edges and blocking invariants**: "Blocking edges" are defined as edges that cannot be further pushed down to lower levels without violating the size invariants. By maintaining the invariant that each cluster has at least one blocking edge, it is ensured that the diameter of the clustered graph does not exceed 2.
- **Parallel graph search**: Using the blocking invariants, efficient parallel graph search can be carried out within a low time - depth.
- **Batch update processing**: Through carefully designed data structures and algorithms, a low time - depth and work complexity can be maintained during batch updates.

In short, this paper aims to bridge the gap between theory and practice in dynamic graph connectivity algorithms and provides an efficient and practical solution.