Abstract:In recent years, there has been increasing interest in adopting published neural retrieval models learned from corpora for text retrieval. Although these models achieve excellent retrieval performance, in terms of popular accuracy metrics, on datasets they have been trained, their performance on new text data might degrade. To obtain the desired retrieval performance on both the data used in training and the latest data collected after training, the simple approach of learning a new model from both datasets is not always feasible since the annotated dataset used in training is often not published along with the learned model. Knowledge amalgamation (KA) is an emerging technique to deal with this problem of inaccessibility of data used in previous training. KA learns a new model (called a student model) from new data by reusing (called amalgamating) a number of trained models (called teacher models) instead of accessing the teachers' original training data. However, in order to efficiently learn an accurate student model, the classical KA approach requires manual selection of an appropriate subset of teacher models for amalgamation. This manual procedure for selecting teacher models prevents the classical KA from being scaled to retrieval tasks for which a large number of candidate teacher models are ready to be reused. This paper presents Arm, an intelligent system for efficiently learning a neural retrieval model with the desired accuracy on incoming data by automatically amalgamating a subset of teacher models (called a teacher model combination or simply combination ) among a large number of teacher models. o filter combinations that fail to produce accurate student models, Arm employs Bayesian optimization to derive an accuracy prediction model based on sampled amalgamation tasks. Then, Arm uses the derived prediction model to exclude unqualified combinations without training the rest combinations. To speed up training, Arm introduces a cost model that picks the teacher model combination with the minimal training cost among all qualified teacher model combinations to produce the final student model. This paper will demonstrate the major workflow of Arm and present the produced student models to users.

A Cooperative Neural Information Retrieval Pipeline with Knowledge Enhanced Automatic Query Reformulation

Arm: Efficient Learning of Neural Retrieval Models with Desired Accuracy by Automatic Knowledge Amalgamation

Mixed-initiative Query Rewriting in Conversational Passage Retrieval

Drop Redundant, Shrink Irrelevant: Selective Knowledge Injection for Language Pretraining

Generative Query Reformulation for Effective Adhoc Search

Translating Embeddings For Modeling Query Reformulation

RIN: Reformulation Inference Network for Context-Aware Query Suggestion.

Multimodal Reranking for Knowledge-Intensive Visual Question Answering

Learning to Reformulate the Queries on the WEB

Learning Better Representations for Neural Information Retrieval with Graph Information

Agent4Ranking: Semantic Robust Ranking via Personalized Query Rewriting Using Multi-agent LLM

A Deep Look into neural ranking models for information retrieval

Incorporating Explicit Knowledge in Pre-trained Language Models for Passage Re-ranking

Improving Biomedical Information Retrieval with Neural Retrievers

Generative Query Reformulation Using Ensemble Prompting, Document Fusion, and Relevance Feedback

Tuning Query Reformulator with Fine-Grained Relevance Feedback

Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering

Knowledge-augmented Self-training of A Question Rewriter for Conversational Knowledge Base Question Answering.

Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Models

CorpusBrain: Pre-train a Generative Retrieval Model for Knowledge-Intensive Language Tasks

Towards a Better Understanding of Query Reformulation Behavior in Web Search