Abstract: Cardinality estimation (CardEst) plays a significant role in generating high-quality query plans for a query optimizer in DBMS. In the last decade, an increasing number of advanced CardEst methods (especially ML-based) have been proposed with outstanding estimation accuracy and inference latency. However, there exists no study that systematically evaluates the quality of these methods and answer the fundamental problem: to what extent can these methods improve the performance of query optimizer in real-world settings, which is the ultimate goal of a CardEst method. In this paper, we comprehensively and systematically compare the effectiveness of CardEst methods in a real DBMS. We establish a new benchmark for CardEst, which contains a new complex realworld dataset STATS and a diverse query workload STATS-CEB. We integrate multiple most representative CardEst methods into an open-source database system PostgreSQL, and comprehensively evaluate their true effectiveness in improving query plan quality, and other important aspects affecting their applicability, ranging from inference latency, model size, and training time, to update efficiency and accuracy. We obtain a number of key findings for the CardEst methods, under different data and query settings. Furthermore, we find that the widely used estimation accuracy metric(Q-Error) cannot distinguish the importance of different sub-plan queries during query optimization and thus cannot truly reflect the query plan quality generated by CardEst methods. Therefore, we propose a new metric P-Error to evaluate the performance of CardEst methods, which overcomes the limitation of Q-Error and is able to reflect the overall end-to-end performance of CardEst methods. We have made all of the benchmark data and evaluation code publicly available at https://github.com/Nathaniel-Han/Endto-End-CardEst-Benchmark.

Skyband cardinality estimation based on the inclusion-exclusion principle

Reverse k-skyband query based on reuse technology

Effective Space Usage Estimation for Sliding-Window Skybands

Skyline Diagram: Efficient Space Partitioning for Skyline Queries

On Efficient k-Skyband Query Processing over Incomplete Data.

Learned Cardinality Estimation: A Design Space Exploration and A Comparative Evaluation

Processing K-Skyband, Constrained Skyline, and Group-by Skyline Queries on Incomplete Data

Probabilistic <Emphasis Type="Italic">k</Emphasis>-Skyband Operator over Sliding Windows

Computing Skyline Probabilities on Uncertain Time Series.

CardSketch: Shift Attention for Network-wide Cardinality Telemetry

Effective Skyline Cardinality Estimation on Data Streams

Fundamental Limits and Optimization of Multiband Delay Estimation in OFDM Systems

On efficient reverse k-skyband query processing

Cardinality Estimation in DBMS

Skyline-Join in Distributed Databases

Cardinality Estimation in DBMS: A Comprehensive Benchmark Evaluation.

On processing reverse k-skyband and ranked reverse skyline queries

Virtual self-adaptive bitmap for online cardinality estimation

Adaptive Cardinality Estimation

Bandwidth Optimisation and Parameter Analysis at Two Adjacent Intersections Based on Set Operations

ByteCard: Enhancing ByteDance's Data Warehouse with Learned Cardinality Estimation