Montage: A General System for Buffered Durably Linearizable Data Structures

Haosen Wen,Wentao Cai,Mingzhe Du,Louis Jenkins,Benjamin Valpey,Michael L. Scott

DOI: https://doi.org/10.48550/arXiv.2009.13701

2020-09-29

Abstract:The recent emergence of fast, dense, nonvolatile main memory suggests that certain long-lived data might remain in its natural pointer-rich format across program runs and hardware reboots. Operations on such data must be instrumented with explicit write-back and fence instructions to ensure consistency in the wake of a crash. Techniques to minimize the cost of this instrumentation are an active topic of research. We present what we believe to be the first general-purpose approach to building buffered durably linearizable persistent data structures, and a system, Montage, to support that approach. Montage is built on top of the Ralloc nonblocking persistent allocator. It employs a slow-ticking epoch clock, and ensures that no operation appears to span an epoch boundary. It also arranges to persist only that data minimally required to reconstruct the structure after a crash. If a crash occurs in epoch $e$, all work performed in epochs $e$ and $e-1$ is lost, but work from prior epochs is preserved. We describe the implementation of Montage, argue its correctness, and report unprecedented throughput for persistent queues, sets/mappings, and general graphs.

Distributed, Parallel, and Cluster Computing,Data Structures and Algorithms,Performance

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is: how to construct a general method to create persistent data structures with buffered persistent linearization to ensure data consistency and recoverability after system crashes. Specifically, the paper proposes the Montage system, which, by introducing an epoch clock with slow ticks, ensures that no operation crosses epoch boundaries and only persists the minimum necessary data to reconstruct the structure after a crash. This solves the performance overhead problems in existing methods caused by frequent write - backs and fence instructions, while ensuring the consistency and correctness of the data structure before and after a crash.

### Key Problem Summary:
1. **Persistence Consistency**: How to ensure that the data structure can maintain a consistent state after system crashes.
2. **Performance Optimization**: How to reduce the performance overhead brought by persistence operations, especially in cases where frequent write - backs and fence instructions are required.
3. **Generality**: How to design a general system that can support multiple persistent data structures, not just specific types of data structures.

### Contributions of the Montage System:
- **Buffered Persistent Linearization**: Montage ensures that each operation is completed within an epoch by introducing the epoch mechanism, avoiding cross - epoch operations, thus simplifying the persistence logic.
- **Minimizing Persistence Overhead**: Montage only persists the minimum necessary data, reducing unnecessary write - back operations and improving performance.
- **Generality**: Montage can be applied not only to sets and maps, but also to any abstract data type that can be represented by a graph.

### Formula Representation:
When describing persistence operations, the paper involves some key time points and operation sequences. To ensure the accuracy and readability of the formulas, the following are the relevant formulas:

- Let \(e\) be the current epoch number.
- Operation \(O_i\) starts within epoch \(e\) and ends within epoch \(e + 1\).
- If a crash occurs within epoch \(e\), all the work within epoch \(e\) and \(e- 1\) will be lost, but the data of previous epochs will be retained.

\[
\text{If a crash occurs in epoch }e,\text{ all the work within epoch }e\text{ and }e - 1\text{ will be lost, but the work before epoch }e-2\text{ will be retained.}
\]

In this way, Montage ensures the correctness and efficiency of the data structure after a crash.