Abstract:Retrieving and extracting knowledge from extensive research documents and large databases presents significant challenges for researchers, students, and professionals in today's information-rich era. Existing retrieval systems, which rely on general-purpose Large Language Models (LLMs), often fail to provide accurate responses to domain-specific inquiries. Additionally, the high cost of pretraining or fine-tuning LLMs for specific domains limits their widespread adoption. To address these limitations, we propose a novel methodology that combines the generative capabilities of LLMs with the fast and accurate retrieval capabilities of vector databases. This advanced retrieval system can efficiently handle both tabular and non-tabular data, understand natural language user queries, and retrieve relevant information without fine-tuning. The developed model, Generative Text Retrieval (GTR), is adaptable to both unstructured and structured data with minor refinement. GTR was evaluated on both manually annotated and public datasets, achieving over 90% accuracy and delivering truthful outputs in 87% of cases. Our model achieved state-of-the-art performance with a Rouge-L F1 score of 0.98 on the MSMARCO dataset. The refined model, Generative Tabular Text Retrieval (GTR-T), demonstrated its efficiency in large database querying, achieving an Execution Accuracy (EX) of 0.82 and an Exact-Set-Match (EM) accuracy of 0.60 on the Spider dataset, using an open-source LLM. These efforts leverage Generative AI and In-Context Learning to enhance human-text interaction and make advanced AI capabilities more accessible. By integrating robust retrieval systems with powerful LLMs, our approach aims to democratize access to sophisticated AI tools, improving the efficiency, accuracy, and scalability of AI-driven information retrieval and database querying.

Towards More Effective Table-to-Text Generation: Assessing In-Context Learning and Self-Evaluation with Open-Source Models

Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study

Bring Your Own Data! Self-Supervised Evaluation for Large Language Models

CT-Eval: Benchmarking Chinese Text-to-Table Performance in Large Language Models

Collaborative Evaluation: Exploring the Synergy of Large Language Models and Humans for Open-ended Generation Evaluation

TableLlama: Towards Open Large Generalist Models for Tables

Discovering Language Model Behaviors with Model-Written Evaluations

Open Source Language Models Can Provide Feedback: Evaluating LLMs' Ability to Help Students Using GPT-4-As-A-Judge

MATEval: A Multi-Agent Discussion Framework for Advancing Open-Ended Text Evaluation

Towards Table-to-Text Generation with Pretrained Language Model: A Table Structure Understanding and Text Deliberating Approach

Reference-Guided Verdict: LLMs-as-Judges in Automatic Evaluation of Free-Form Text

Generating Tables from the Parametric Knowledge of Language Models

Automated Data Visualization from Natural Language via Large Language Models: An Exploratory Study

Leveraging Large Language Models for NLG Evaluation: Advances and Challenges

Teaching Models to Improve on Tape

Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generation

Table-LLM-Specialist: Language Model Specialists for Tables using Iterative Generator-Validator Fine-tuning

Leveraging Prior Experience: An Expandable Auxiliary Knowledge Base for Text-to-SQL

Uncovering Limitations of Large Language Models in Information Seeking from Tables

Enhancing Knowledge Retrieval with In-Context Learning and Semantic Search through Generative AI

Evaluating Large Language Models at Evaluating Instruction Following