Abstract:Query cost estimation, which is to estimate the query plan cost and query execution cost, is of utmost importance to query optimizers. Query plan cost estimation heavily relies on accurate cardinality estimation, and query execution cost estimation gives good hints on query latency, both of which are challenging in database management systems. Despite decades of research, existing studies either over-simplify the models only using histograms and polynomial calculation that leads to inaccurate estimates, or over-complicate them by using cumbersome neural networks with the requirements for large amounts of training data hence poor computational efficiency. Besides, most of the studies ignore the diversity of query plan structures. In this work, we propose a plan-based query cost estimation framework, called Saturn, which can eStimate cardinality and latency accurately and efficiently, for any query plan structures. Saturn first encodes each query plan tree into a compressed vector by using a traversal-based query plan autoencoder to cope with diverse plan structures. The compressed vectors can be leveraged to distinguish different query types, which is highly useful for downstream tasks. Then a pseudo label generator is designed to acquire all cardinality and latency labels with the execution part of the query plans in the training workload, which can significantly reduce the overhead of collecting the real cardinality and latency labels. Finally, a chain-wise transfer learning module is proposed to estimate the cardinality and latency of the query plan in a pipeline paradigm, which further enhances the efficiency. An extensive empirical study on benchmark data offers evidence that Saturn outperforms the state-of-the-art proposals in terms of accuracy, efficiency, and generalizability for query cost estimation.

Precision Meets Resilience: Cross-Database Generalization with Uncertainty Quantification for Robust Cost Estimation

DACE: A Database-Agnostic Cost Estimator

Can Uncertainty Quantification Enable Better Learning-based Index Tuning?

PRICE: A Pretrained Model for Cross-Database Cardinality Estimation

A Generalized Multi-Fidelity Simulation Method Using Sparse Polynomial Chaos Expansion

Cost-based or Learning-based? A Hybrid Query Optimizer for Query Plan Selection

Database Query Cost Prediction Using Recurrent Neural Network

Cleaning Uncertain Data with Crowdsourcing - a General Model with Diverse Accuracy Rates

Rethinking Learned Cost Models: Why Start from Scratch?

Unified Uncertainty Estimation for Cognitive Diagnosis Models

Cost-Based or Learning-Based?

A Survey on Advancing the DBMS Query Optimizer: Cardinality Estimation, Cost Model, and Plan Enumeration

QCFE: An efficient Feature engineering for query cost estimation

An End-to-End Learning-based Cost Estimator

Adaptive Cost Model for Query Optimization

BSM loss: A superior way in modeling aleatory uncertainty of fine_grained classification

SURE: SUrvey REcipes for building reliable and robust deep networks

Efficient Learning with Pseudo Labels for Query Cost Estimation

Neural Message Passing for Objective-Based Uncertainty Quantification and Optimal Experimental Design.

A Cost-Aware Approach to Adversarial Robustness in Neural Networks

Confidence-Driven Design Optimization Using Gaussian Process Metamodeling With Insufficient Data