A new data-intensive parallel processing framework for spatial data

Dong Zhao,Yang Gu,Zhenchun Huang
DOI: https://doi.org/10.1007/978-3-319-01766-2_43
2014-01-01
Abstract:The explosive increase of scientific data brings in the “Fourth Paradigm” research method by Jim Gray. In order to accelerate the processing speed for these big data, parallel distributed processing is needed. As the data-intensive computing requires high throughput of IO, the data transfer from different node should be cut down as much as possible. Current technologies focus more on the framework for local reliable network with homogeneous resources, but the parallel processing framework for scientific data-intensive problems such as spatial data shared with the Internet and queried by semantics is not fully studied. In this article, we proposed a new data-intensive parallel processing framework for spatial data—Robinia DSSSD (Distributed Storage and Service for Spatial Data), which provides the flexible ability to support data distribution and allocation across the Internet, and semantics query. Experiments shows that Robinia DSSSD can achieve good acceleration with low overhead, and it can well support data-intensive computing.
What problem does this paper attempt to address?