SQLPrompt: In-Context Text-to-SQL with Minimal Labeled Data

Ruoxi Sun,Sercan Ö. Arik,Rajarishi Sinha,Hootan Nakhost,Hanjun Dai,Pengcheng Yin,Tomas Pfister

2023-11-06

Abstract:Text-to-SQL aims to automate the process of generating SQL queries on a database from natural language text. In this work, we propose "SQLPrompt", tailored to improve the few-shot prompting capabilities of Text-to-SQL for Large Language Models (LLMs). Our methods include innovative prompt design, execution-based consistency decoding strategy which selects the SQL with the most consistent execution outcome among other SQL proposals, and a method that aims to improve performance by diversifying the SQL proposals during consistency selection with different prompt designs ("MixPrompt") and foundation models ("MixLLMs"). We show that \emph{SQLPrompt} outperforms previous approaches for in-context learning with few labeled data by a large margin, closing the gap with finetuning state-of-the-art with thousands of labeled data.

Computation and Language

What problem does this paper attempt to address?

The problem this paper attempts to address is the automated generation of SQL queries from natural language (Text-to-SQL) using a small amount of labeled data. Specifically, the paper proposes a method called "SQLPrompt," which aims to enhance the Text-to-SQL capabilities of large language models (LLMs) under few-shot prompting. The paper focuses on the following aspects: 1. **Innovative Prompt Design**: Guiding the model to generate diverse SQL queries through different prompt designs. 2. **Execution-based Consistency Decoding Strategy**: Improving accuracy by selecting the SQL query with the most consistent execution results. 3. **Methods for Diverse SQL Proposals**: Enhancing the diversity of SQL proposals by combining different prompt designs ("MixPrompt") and different base models ("MixLLMs"). The purpose of these methods is to improve the performance of the Text-to-SQL task with a small amount of labeled data, thereby reducing the reliance on large amounts of labeled data, lowering the requirements for adapting to data, and reducing the risks of overfitting and poor generalization.

SQLPrompt: In-Context Text-to-SQL with Minimal Labeled Data

Enhancing Few-shot Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies

PET-SQL: A Prompt-Enhanced Two-Round Refinement of Text-to-SQL with Cross-consistency

How to Prompt LLMs for Text-to-SQL: A Study in Zero-shot, Single-domain, and Cross-domain Settings

Retrieval-augmented GPT-3.5-based Text-to-SQL Framework with Sample-aware Prompting and Dynamic Revision Chain

Prompting GPT-3.5 for Text-to-SQL with De-semanticization and Skeleton Retrieval

ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought

CoE-SQL: In-Context Learning for Multi-Turn Text-to-SQL with Chain-of-Editions

MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generation

SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended)

Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation

RH-SQL: Refined Schema and Hardness Prompt for Text-to-SQL

Divide and Prompt: Chain of Thought Prompting for Text-to-SQL

RB-SQL: A Retrieval-based LLM Framework for Text-to-SQL

Adapt and Decompose: Efficient Generalization of Text-to-SQL via Domain Adapted Least-To-Most Prompting

DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction

Domain-Specific Few-Shot Table Prompt Question Answering via Contrastive Exemplar Selection

SEA-SQL: Semantic-Enhanced Text-to-SQL with Adaptive Refinement

SelECT-SQL: Self-correcting ensemble Chain-of-Thought for Text-to-SQL

SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL

Large Language Models Know Your Contextual Search Intent: A Prompting Framework for Conversational Search