Abstract:Although advancements in deep learning have significantly enhanced the recommendation accuracy of deep recommendation models, these methods still suffer from low recommendation efficiency. Recently proposed tree-based deep recommendation models alleviate the problem by directly learning tree structure and representations under the guidance of recommendation objectives. To guarantee the effectiveness of beam search for recommendation accuracy, these models strive to ensure that the tree adheres to the max-heap assumption, where a parent node's preference should be the maximum among its children's preferences. However, they employ a one-versus-all strategy, framing the training task as a series of independent binary classification objectives for each node, which limits their ability to fully satisfy the max-heap assumption. To this end, we propose a Deep Tree-based Retriever (DTR for short) for efficient recommendation. DTR frames the training task as a softmax-based multi-class classification over tree nodes at the same level, enabling explicit horizontal competition and more discriminative top-k selection among them, which mimics the beam search behavior during training. To mitigate the suboptimality induced by the labeling of non-leaf nodes, we propose a rectification method for the loss function, which further aligns with the max-heap assumption in expectation. As the number of tree nodes grows exponentially with the levels, we employ sampled softmax to approximate optimization and thereby enhance efficiency. Furthermore, we propose a tree-based sampling method to reduce the bias inherent in sampled softmax. Theoretical results reveal DTR's generalization capability, and both the rectification method and tree-based sampling contribute to improved generalization. The experiments are conducted on four real-world datasets, validating the effectiveness of the proposed method.

Constructing Tree-based Index for Efficient and Effective Dense Retrieval

Learning To Retrieve: How to Train a Dense Retrieval Model Effectively and Efficiently

Learning Deep Tree-based Retriever for Efficient Recommendation: Theory and Method

ControlRetriever: Harnessing the Power of Instructions for Controllable Retrieval

DynamicRetriever: A Pre-trained Model-based IR System Without an Explicit Index

Optimizing Dense Retrieval Model Training with Hard Negatives.

Learning Balanced Tree Indexes for Large-Scale Vector Retrieval

Efficient Bulk Loading to Accelerate Spatial Keyword Queries

Hybrid Inverted Index is a Robust Accelerator for Dense Retrieval

Research on Streamline Inter-relevant Successive Trees

Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performance

Learning Optimal Tree Models under Beam Search.

DynamicRetriever: A Pre-training Model-based IR System with Neither Sparse nor Dense Index

EHI: End-to-end Learning of Hierarchical Index for Efficient Dense Retrieval

Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval

Adversarial Retriever-Ranker for Dense Text Retrieval.

Joint Optimization of Tree-based Index and Deep Model for Recommender Systems

CD-Tree: A Clustering-Based Dynamic Indexing and Retrieval Approach.

MCR-Tree: An Efficient Index for Multi-dimensional Core Search.

DPTDR: Deep Prompt Tuning for Dense Passage Retrieval

Preprint. Under review