Abstract:In the last decades, the necessity to process massive amounts of textual data fueled the development of compressed text indexes: data structures efficiently answering queries on a given text while occupying space proportional to the compressed representation of the text. A widespread phenomenon in compressed indexing is that more powerful queries require larger indexes. For example, random access, the most basic query, can be supported in $O(\delta\log\frac{n\log\sigma}{\delta\log n})$ space (where $n$ is the text length, $\sigma$ is the alphabet size, and $\delta$ is text's substring complexity), which is the asymptotically smallest space to represent a string, for all $n$, $\sigma$, and $\delta$ (Kociumaka, Navarro, Prezza; IEEE Trans. Inf. Theory 2023). The other end of the hierarchy is occupied by indexes supporting the powerful suffix array (SA) queries. The currently smallest one takes $O(r\log\frac{n}{r})$ space, where $r\geq\delta$ is the number of runs in the BWT of the text (Gagie, Navarro, Prezza; J. ACM 2020). We present a new compressed index that needs only $O(\delta\log\frac{n\log\sigma}{\delta\log n})$ space to support SA functionality in $O(\log^{4+\epsilon} n)$ time. This collapses the hierarchy of compressed data structures into a single point: The space required to represent the text is simultaneously sufficient for efficient SA queries. Our result immediately improves the space complexity of dozens of algorithms, which can now be executed in optimal compressed space. In addition, we show how to construct our index in $O(\delta\text{ polylog } n)$ time from the LZ77 parsing of the text. For highly repetitive texts, this is up to exponentially faster than the previously best algorithm. To obtain our results, we develop numerous techniques of independent interest, including the first $O(\delta\log\frac{n\log\sigma}{\delta\log n})$-size index for LCE queries.

A Sierpinski Triangle Data Structure for Efficient Array Value Update and Prefix Sum Calculation

A Fast Trie Tree Index Construction Algorithm Using Frequency Characteristic

Dynamic "Succincter"

A Sierpinski Triangle Fermion-to-Qubit Transform

A simple and space efficient segment tree implementation

Collapsing the Hierarchy of Compressed Data Structures: Suffix Arrays in Optimal Compressed Space

Optimal prefix-suffix queries with applications

The Wavelet Trie: Maintaining an Indexed Sequence of Strings in Compressed Space

AOT: Pushing the Efficiency Boundary of Main-memory Triangle Listing

An Improved Algorithm for Incremental DFS Tree in Undirected Graphs

RadixKV - A Memory Efficient and High Performance Key-Value Store.

Dynamic Suffix Array in Optimal Compressed Space

Reversed Indexes $\approx$ Values in Wavelet Trees

Enumeration and Succinct Encoding of AVL Trees

Packing R-trees with Space-filling Curves

Fast Triangle Counting

Fully-Functional Static and Dynamic Succinct Trees

BITS-Tree-An Efficient Data Structure for Segment Storage and Query Processing

A High Throughput B+tree for SIMD Architectures

Accelerated Boolean Operations on Triangular Mesh Models Based on Dynamic Spatial Indexing

Packing R-trees with Space-filling Curves: Theoretical Optimality, Empirical Efficiency, and Bulk-loading Parallelizability