Abstract:Federated search, which involves integrating results from multiple independent search engines, will become increasingly pivotal in the context of Retrieval-Augmented Generation pipelines empowering LLM-based applications such as chatbots. These systems often distribute queries among various search engines, ranging from specialized (e.g., PubMed) to general (e.g., Google), based on the nature of user utterances. A critical aspect of federated search is resource selection - the selection of appropriate resources prior to issuing the query to ensure high-quality and rapid responses, and contain costs associated with calling the external search engines. However, current SOTA resource selection methodologies primarily rely on feature-based learning approaches. These methods often involve the labour intensive and expensive creation of training labels for each resource. In contrast, LLMs have exhibited strong effectiveness as zero-shot methods across NLP and IR tasks. We hypothesise that in the context of federated search LLMs can assess the relevance of resources without the need for extensive predefined labels or features. In this paper, we propose ReSLLM. Our ReSLLM method exploits LLMs to drive the selection of resources in federated search in a zero-shot setting. In addition, we devise an unsupervised fine tuning protocol, the Synthetic Label Augmentation Tuning (SLAT), where the relevance of previously logged queries and snippets from resources is predicted using an off-the-shelf LLM and then in turn used to fine-tune ReSLLM with respect to resource selection. Our empirical evaluation and analysis details the factors influencing the effectiveness of LLMs in this context. The results showcase the merits of ReSLLM for resource selection: not only competitive effectiveness in the zero-shot setting, but also obtaining large when fine-tuned using SLAT-protocol.

Know where to go: Make LLM a relevant, responsible, and trustworthy searchers

Know Where to Go: Make LLM a Relevant, Responsible, and Trustworthy Searcher

WeKnow-RAG: An Adaptive Approach for Retrieval-Augmented Generation Integrating Web Search and Knowledge Graphs

Re-Search for The Truth: Multi-round Retrieval-augmented Large Language Models are Strong Fake News Detectors

Leveraging Large Language Models for Relevance Judgments in Legal Case Retrieval

Spiral of Silence: How is Large Language Model Killing Information Retrieval? -- A Case Study on Open Domain Question Answering

Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibility

Large Language Models are Built-in Autoregressive Search Engines

Are LLMs Really Not Knowledgable? Mining the Submerged Knowledge in LLMs' Memory

ReSLLM: Large Language Models are Strong Resource Selectors for Federated Search

Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generators

When Search Engine Services meet Large Language Models: Visions and Challenges

Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong

Adapting LLMs for Efficient, Personalized Information Retrieval: Methods and Implications

Supervised Knowledge Makes Large Language Models Better In-context Learners

R4: Reinforced Retriever-Reorder-Responder for Retrieval-Augmented Large Language Models

Exploring the Best Practices of Query Expansion with Large Language Models

LLMGR: Large Language Model-based Generative Retrieval in Alipay Search

LLMs Know What They Need: Leveraging a Missing Information Guided Framework to Empower Retrieval-Augmented Generation

ARL2: Aligning Retrievers for Black-box Large Language Models via Self-guided Adaptive Relevance Labeling

Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs