Abstract:Cardinality estimation is one of the most important problems in query optimization. Recently, machine learning based techniques have been proposed to effectively estimate cardinality, which can be broadly classified into query-driven and data-driven approaches. Query-driven approaches learn a regression model from a query to its cardinality; while data-driven approaches learn a distribution of tuples, select some samples that satisfy a SQL query, and use the data distributions of these selected tuples to estimate the cardinality of the SQL query. As query-driven methods rely on training queries, the estimation quality is not reliable when there are no high-quality training queries; while data-driven methods have no such limitation and have high adaptivity. In this work, we focus on data-driven methods. A good datadriven model should achieve three optimization goals. First, the model needs to capture data dependencies between columns and support large domain sizes (achieving high accuracy). Second, the model should achieve high inference efficiency, because many data samples are needed to estimate the cardinality (achieving low inference latency). Third, the model should not be too large (achieving a small model size). However, existing data-driven methods cannot simultaneously optimize the three goals. To address the limitations, we propose a novel cardinality estimator FACE, which leverages the Normalizing Flow based model to learn a continuous joint distribution for relational data. FACE can transform a complex distribution over continuous random variables into a simple distribution (e.g., multivariate normal distribution), and use the probability density to estimate the cardinality. First, we design a dequantization method to make data more "continuous". Second, we propose encoding and indexing techniques to handle Like predicates for string data. Third, we propose a Monte Carlo method to efficiently estimate the cardinality. Experimental results show that our method significantly outperforms existing approaches in terms of estimation accuracy while keeping similar latency and model size.

Efficient and Effective Cardinality Estimation for Skyline Family.

Eclipse: Generalizing Knn and Skyline

Dissertation Defense Efficient and Adaptive Skyline Computation

Eclipse: Practicability Beyond Knn and Skyline

Skyline Diagram: Efficient Space Partitioning for Skyline Queries

Effective Skyline Cardinality Estimation on Data Streams

Updateable Data-Driven Cardinality Estimator with Bounded Q-error

ByteCard: Enhancing ByteDance's Data Warehouse with Learned Cardinality Estimation

AutoCE: An Accurate and Efficient Model Advisor for Learned Cardinality Estimation

Cardinality Estimation in DBMS: A Comprehensive Benchmark Evaluation.

One stone, two birds: A lightweight multidimensional learned index with cardinality support

Cardinality Estimation in DBMS

Learned Cardinality Estimation: A Design Space Exploration and A Comparative Evaluation

Adaptive Cardinality Estimation

FLAT: Fast, Lightweight and Accurate Method for Cardinality Estimation

FACE: A Normalizing Flow based Cardinality Estimator

Skyline-Join in Distributed Databases

An Empirical Analysis of Deep Learning for Cardinality Estimation

CardBench: A Benchmark for Learned Cardinality Estimation in Relational Databases

An End-to-End Learning-based Cost Estimator

CardOOD: Robust Query-driven Cardinality Estimation under Out-of-Distribution