Approximate Nearest Neighbor Search on High Dimensional Data — Experiments, Analyses, and Improvement
Wen Li,Ying Zhang,Yifang Sun,Wei Wang,Mingjie Li,Wenjie Zhang,Xuemin Lin
DOI: https://doi.org/10.1109/tkde.2019.2909204
IF: 9.235
2020-08-01
IEEE Transactions on Knowledge and Data Engineering
Abstract:Nearest neighbor search is a fundamental and essential operation in applications from many domains, such as databases, machine learning, multimedia, and computer vision. Because exact searching results are not efficient for a high-dimensional space, a lot of efforts have turned to approximate nearest neighbor search. Although many algorithms have been continuously proposed in the literature each year, there is no comprehensive evaluation and analysis of their performance. In this paper, we conduct a comprehensive experimental evaluation of many state-of-the-art methods for approximate nearest neighbor search. Our study (1) is cross-disciplinary (i.e., including 19 algorithms in different domains, and from practitioners) and (2) has evaluated a diverse range of settings, including 20 datasets, several evaluation metrics, and different query workloads. The experimental results are carefully reported and analyzed to understand the performance results. Furthermore, we propose a new method that achieves both high query efficiency and high recall empirically on majority of the datasets under a wide range of settings.
computer science, information systems, artificial intelligence,engineering, electrical & electronic