Abstract:This study presents an innovative enhancement to retrieval-augmented generation (RAG) systems by seamlessly integrating fine-tuned large language models (LLMs) with vector databases. This integration capitalizes on the combined strengths of structured data retrieval and the nuanced comprehension provided by advanced LLMs. Central to our approach are the LoRA and QLoRA methodologies, which stand at the forefront of model refinement through parameter-efficient fine-tuning and memory optimization. A novel feature of our research is the incorporation of user feedback directly into the training process, ensuring the model's continuous adaptation to user expectations and thus, improving its performance and applicability. Additionally, we introduce a Quantized Influence Measure (QIM) as an innovative "AI Judge" mechanism to enhance the precision of result selection, further refining the system's accuracy. Accompanied by an executive diagram and a detailed algorithm for fine-tuning QLoRA, our work provides a comprehensive framework for implementing these advancements within chatbot technologies. This research contributes significant insights into LLM optimization for specific uses and heralds new directions for further development in retrieval-augmented models. Through extensive experimentation and analysis, our findings lay a robust foundation for future advancements in chatbot technology and retrieval systems, marking a significant step forward in the creation of more sophisticated, precise, and user-centric conversational AI systems.

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve The paper aims to address the increasingly severe issue of homeless families in the United States, particularly the lack of communication channels and limited critical resources in homeless shelters. Specifically, the paper proposes a web-based chatbot interface based on Large Language Models (LLM) and Generative Artificial Intelligence (GAI) technologies to significantly enhance the resources and communication channels for homeless shelters and their service recipients. ### Main Contributions 1. **Enhanced Retrieval-Augmented Generation (RAG) System**: - Improved communication efficiency by seamlessly integrating fine-tuned Large Language Models (LLM) with traditional vector databases. - Utilized LoRA and QLoRA methods for parameter-efficient fine-tuning and memory optimization. - Introduced Quantitative Impact Measurement (QIM) as an "AI Judge" mechanism to improve the accuracy and relevance of query result selection. 2. **Fine-Tuning Strategies**: - Detailed the fine-tuning methods of LoRA and QLoRA, including parameter adjustments and optimization processes. - Proposed a specific fine-tuning algorithm that optimizes model performance by iteratively adjusting attention dimensions, LoRA scaling factors, and dropout probabilities. 3. **Quantitative Impact Measurement (QIM)**: - Introduced a new quantitative impact measurement method to assess the similarity between queries and references. - Experimentally validated that QIM outperforms cosine similarity in high-dimensional data, especially when handling large datasets. ### Experiments and Applications - **Dataset**: Custom dataset collected from the Youth Spirit Artworks (YSA) Tiny House Empowerment Village website. - **Model**: Fine-tuned LLM model publicly available on Hugging Face. - **Application**: A Hugging Face-based chatbot application for practical testing and deployment. Through these innovations, the paper not only provides new technical directions for the AI community but also offers better access to information and resources for low-income and underserved populations.

A Fine-tuning Enhanced RAG System with Quantized Influence Measure as AI Judge

Enhancing Q&A with Domain-Specific Fine-Tuning and Iterative Reasoning: A Comparative Study

Evaluating the Retrieval Component in LLM-Based Question Answering Systems

RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation

IM-RAG: Multi-Round Retrieval-Augmented Generation Through Learning Inner Monologues

A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models

Boosting Conversational Question Answering with Fine-Grained Retrieval-Augmentation and Self-Check

RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs

Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generation

LA-RAG:Enhancing LLM-based ASR Accuracy with Retrieval-Augmented Generation

Reinforcement Learning for Optimizing RAG for Domain Chatbots

Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Bases

Towards Understanding Retrieval Accuracy and Prompt Quality in RAG Systems

LoRA-LiteE: A Computationally Efficient Framework for Chatbot Preference-Tuning

RAG4ITOps: A Supervised Fine-Tunable and Comprehensive RAG Framework for IT Operations and Maintenance

Invar-RAG: Invariant LLM-aligned Retrieval for Better Generation

Investigating the performance of Retrieval-Augmented Generation and fine-tuning for the development of AI-driven knowledge-based systems

Towards Optimizing and Evaluating a Retrieval Augmented QA Chatbot using LLMs with Human in the Loop

AssistRAG: Boosting the Potential of Large Language Models with an Intelligent Information Assistant

Blended RAG: Improving RAG (Retriever-Augmented Generation) Accuracy with Semantic Search and Hybrid Query-Based Retrievers

RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing