Abstract:Semantic indexing is a popular technique used to access and organize large amounts of unstructured text data. We describe an optimized implementation of semantic indexing and document search on manycore GPU platforms. We observed that a parallel implementation of semantic indexing on a 128-core Tesla C870 GPU is only 2.4X faster than a sequential implementation on an Intel Xeon 2.4GHz processor. We ascribe the less than spectacular speedup to a mismatch in the workload characteristics of semantic indexing and the unique architectural features of GPUs. Compared to the regular numerical computations that have been ported to GPUs with great success, our semantic indexing algorithm (the recently proposed Supervised Semantic Indexing algorithm called SSI) has interesting characteristics -- the amount of parallelism in each training instance is data-dependent, and each iteration involves the product of a dense matrix with a sparse vector, resulting in random memory access patterns. As a result, we observed that the baseline GPU implementation significantly under-utilizes the hardware resources (processing elements and memory bandwidth) of the GPU platform. However, the SSI algorithm also demonstrates unique characteristics, which we collectively refer to as the "forgiving nature" of the algorithm. These unique characteristics allow for novel optimizations that do not strive to preserve numerical equivalence of each training iteration with the sequential implementation. In particular, we consider best-effort computing techniques, such as dependency relaxation and computation dropping, to suitably alter the workload characteristics of SSI to leverage the unique architectural features of the GPU. We also show that the realization of dependency relaxation and computation dropping concepts on a GPU is quite different from how one would implement these concepts on a multicore CPU, largely due to the distinct architectural features supported by a GPU. Our new techniques dramatically enhance the amount of parallel workload, leading to much higher performance on the GPU. By optimizing data transfers between CPU and GPU, and by reducing GPU kernel invocation overheads, we achieve further performance gains. We evaluated our new GPU-accelerated implementation of semantic document search on a database of over 1.8 million documents from Wikipedia. By applying our novel performance-enhancing strategies, our GPU implementation on a 128-core Tesla C870 achieved a 5.5X acceleration as compared to a baseline parallel implementation on the same GPU. Compared to a baseline parallel TBB implementation on a dual-socket quad-core Intel Xeon multicore CPU (8-cores), the enhanced GPU implementation is 11X faster. Compared to a parallel implementation on the same multi-core CPU that also uses data dependency relaxation and dropping computation techniques, our enhanced GPU implementation is 5X faster.

Billion-Scale Similarity Search with GPUs

A social network-aware top-N recommender system using GPU.

GTS: GPU-based Tree Index for Fast Similarity Search

BANG: Billion-Scale Approximate Nearest Neighbor Search using a Single GPU

Vector and Line Quantization for Billion-scale Similarity Search on GPUs

Similarity search in the blink of an eye with compressed indices

Best-effort semantic document search on GPUs

Hierarchical quantization for billion-scale similarity retrieval on GPUs

Fast Document Cosine Similarity Self-Join on GPUs.

Efficient Large-scale Approximate Nearest Neighbor Search on the GPU

Efficient Feature Detection and Effective Post-Verification for Large Scale Near-Duplicate Image Search

Extreme-scale many-against-many protein similarity search

Learning-based Efficient Graph Similarity Computation via Multi-Scale Convolutional Set Matching

Towards Distributed Node Similarity Search on Graphs

A Generic Inverted Index Framework for Similarity Search on the GPU - Technical Report

Large Scale Nearest Neighbors Search Based on Neighborhood Graph.

Semi-Convex Hull Tree: Fast Nearest Neighbor Queries for Large Scale Data on GPUs

GPU-utilized Composite Quantization for Large-scale High-dimension ANN Search

CAGRA: Highly Parallel Graph Construction and Approximate Nearest Neighbor Search for GPUs

GPU Accelerated Cascade Hashing Image Matching for Large Scale 3D Reconstruction

Hybrid CPU-GPU Framework for Network Motifs