Efficient Processing of the SkyEXP Query Over Big Data.
Zhenhua Huang,Chang Yu,Yong Tang,Yunwen Chen,Shuhua Zhang,Zhonghua Zheng
DOI: https://doi.org/10.1007/978-981-13-0893-2_40
2017-01-01
Abstract:Skyline query processing has recently received a lot of attention in the big data analysis community. However, in most real applications, the skyline result can not satisfy the needs of users. In this paper, we propose a novel type of skyEXP query to more efficiently analyze and explore the data. The skyEXP query on the subspace V divides the input data M into w separate subsets SE1 (M, V),…, SEw (M, V) such that an object p belongs to SEi (M, V) if it is not dominated by any other objects on V except for those in SE1 (M, V),…, SEi−1(M, V) where i ∈ [1, w]. In order to fast implement the proposed query over big data, an efficient parallel algorithm SQMRM (the SkyEXP Query using Map-Reduce Model) which utilizes the map-reduce framework is presented. Detailed theoretical analyses and extensive experiments demonstrate that our SQMRM algorithm is both efficient and effective.