QBS-Tree: A Spatial Index with High Update Efficiency for Real-Time Processing System
Zonglei Zhang,Junhua Fang,Wei Chen,Detian Zhang,An Liu,Zhixu Li
DOI: https://doi.org/10.1109/hpcc/smartcity/dss.2019.00179
2019-01-01
Abstract:Along with the popularization of big data applications, real-time data analysis is playing an increasingly important role in data analytic applications. The distributed parallel processing framework is a good choice for real-time big data processing, since it provides an effective solution which facilitates processing of massive data stream with low latency. Furthermore, among the real-time applications, location based service is recognized as the most challenging one, especially under expectation on fast response to distribution change and minimal resource consumption. We find that the fundamental reason for this is that the index performance is a stumbling block. Specifically, the original intention of index design is to speed up retrieval speed; in other words, the index is more useful for operations that have more queries than updates. However, in the distributed parallel processing framework, each incoming tuple is both a query request and an update request. In this paper, we propose a new tree index, quad balanced space tree called QBS-tree, to solve this problem, with practical algorithms to support dynamic index items assignment for distributed parallel processing framework. Specifically, make tree index's tolerability to the index items' unbalanced distribution in space be more resilient through a configurable balance factor. In addition, through the lazy update mechanism, let some updates offset each other and the upper node's update covers the underlying node's. Experiments show that QBS-tree responds to a new tuple 1300 times faster than the traditional tree index.