Abstract:Equivalence checking of two SQL queries is an intractable problem encountered in diverse contexts ranging from grading student submissions in a DBMS course to debugging query rewriting rules in an optimizer, and many more. While a lot of progress has been made in recent years in developing practical solutions for this problem, the existing methods can handle only a small subset of SQL, even for bounded equivalence checking. They cannot support sophisticated SQL expressions one encounters in practice. At the same time, large language models (LLMs) -- such as GPT-4 -- have emerged as power generators of SQL from natural language specifications. This paper explores whether LLMs can also demonstrate the ability to reason with SQL queries and help advance SQL equivalence checking. Towards this, we conducted a detailed evaluation of several LLMs over collections with SQL pairs of varying levels of complexity. We explored the efficacy of different prompting techniques, the utility of synthetic examples & explanations, as well as logical plans generated by query parsers. Our main finding is that with well-designed prompting using an unoptimized SQL Logical Plan, LLMs can perform equivalence checking beyond the capabilities of current techniques, achieving nearly 100% accuracy for equivalent pairs and up to 70% for non-equivalent pairs of SQL queries. While LLMs lack the ability to generate formal proofs, their synthetic examples and human-readable explanations offer valuable insights to students (& instructors) in a classroom setting and to database administrators (DBAs) managing large database installations. Additionally, we also show that with careful fine-tuning, we can close the performance gap between smaller (and efficient) models and larger models such as GPT, thus paving the way for potential LLM-integration in standalone data processing systems.

GEqO: ML-Accelerated Semantic Equivalence Detection

Semantic SQL -- Combining and optimizing semantic predicates in SQL

A Parallel Function Evaluation Approach for Solution to Large-Scale Equation-Oriented Models.

Semantic Equivalence of e-Commerce Queries

LLM-SQL-Solver: Can LLMs Determine SQL Equivalence?

Automated verification of query equivalence using satisfiability modulo theories

Exploring the Use of LLMs for SQL Equivalence Checking

Datalog-based Scalable Semantic Diffing of Concurrent Programs

Optimizing analytic data flows for multiple execution engines

QED: A Powerful Query Equivalence Decider for SQL

Quantifying Semantic Query Similarity for Automated Linear SQL Grading: A Graph-based Approach

UQE: A Query Engine for Unstructured Databases

Based on the Gemini Large-scale Model, Enhanced Accuracy in Semantic Similarity Detection With the Ernie Model

Veer: Verifying Equivalence of Dataflow Versions in Iterative Data Analytics (Extended Version)

Axiomatic Foundations and Algorithms for Deciding Semantic Equivalences of SQL Queries

A Novel Similarity Metric with Application to Big Process Data Analytics

Integration of Semantic and Topological Structural Similarity Comparison for Entity Alignment without Pre-Training

ML-AQP: Query-Driven Approximate Query Processing based on Machine Learning

Aggregation Consistency Errors in Semantic Layers and How to Avoid Them

An Efficient Industrial Big-Data Engine