Abstract:In recent years, there has been a growing interest in the application of deep reinforcement learning (DRL) techniques on query execution plan generation. Although current DRL-based query optimizers achieve competitive performance against traditional methods on specific query workloads, these methods encounter issues when generalizing to workloads unseen during training. Thus, we propose GLO to address the limitations and step towards generalized learned query optimization. First, rather than using ungeneralizable table-specific one-hot labels in almost all existing work, GLO relies on statistical information of the well-established underlying DBMS along with table patterns extracted via a clustering algorithm, enabling GLO to enhance generalization in different scenarios. Second, GLO improves the information capture of plans by integrating Transformer layers into the DRL value model, empowering the model's capability to handle diverse queries with deeper networks and more parameters in plan generation. In addition, GLO allows the injection of cost estimations from the DBMS as external knowledge for better generalization. Third, GLO recognizes and replaces disastrously poor plans by making comparisons between generated plans and those produced by the DBMS. We establish our experiments on composite workloads that combine various query sets including JOB, Extended JOB, TPC-DS, and Stack. The results demonstrate that GLO outperforms previous state-of-the-art learned optimizers, with a speed 1.4x faster than LOGER and 2.1x faster than Balsa on TPC-DS when TPC-DS queries are completely unknown during training. To the best of our knowledge, GLO is the first learned optimizer that directly generates plans while possessing the preliminary generalization ability across different query workloads.

gFOV: A Full-Stack SPARQL Query Optimizer & Plan Visualizer

Efficient Execution of SPARQL Queries with OPTIONAL and UNION Expressions

FOSS: A Self-Learned Doctor for Query Optimizer

A Modular Graph-Native Query Optimization Framework

ROSIE: Runtime Optimization of SPARQL Queries over RDF Using Incremental Evaluation.

ROSIE: Runtime Optimization of SPARQL Queries Using Incremental Evaluation.

PRAGUE: Towards Blending Practical Visual Subgraph Query Formulation and Query Processing

Gcbo: A Cost-based Optimizer for Graph Databases.

Gstore: a Graph-Based SPARQL Query Engine

GSmart: An Efficient SPARQL Query Engine Using Sparse Matrix Algebra -- Full Version

Towards a Converged Relational-Graph Optimization Framework

SPARQL Multi-Query Optimization.

Gtop: an Efficient SPARQL Query Engine.

Towards Query Optimization for SPARQL Property Paths

Redesign of the Gstore System

The Odyssey Approach for Optimizing Federated SPARQL Queries

Jovis: A Visualization Tool for PostgreSQL Query Optimizer

SOFA: An Extensible Logical Optimizer for UDF-heavy Dataflows

SPARQL Query Parallel Processing: A Survey

GLO: Towards Generalized Learned Query Optimization

LOGER: A Learned Optimizer Towards Generating Efficient and Robust Query Execution Plans