Research on Key Technology of Spatial Analysis of Large-scale Geospatial Data in Cloud
Yi-yang SHAO,Wei-dong BAO,Xiao-min ZHU
DOI: https://doi.org/10.12783/dtcse/cst2017/12597
2017-01-01
DEStech Transactions on Computer Science and Engineering
Abstract:Geospatial data not merely affects to people's daily life, but also plays a key role in national defense construction and economic development. As geospatial data is gradually gaining scale and complexity with the emergence of the big data era, the conventional spatial data management solution, represented by the combination of ArcSDE and Oracle based on relational database, can no longer satisfy those more demanding needs of data management and analysis. In the basis of exploring the essence, features, and advantages of cloud computing, this paper pays special attention to the Hadoop Distributed File System (HDFS) and its depending framework MapReduce which supports distributed computing given a large data sets. Aiming to solve the difficulty of earthquake frequency statistics across different province of China, we collected a large volume of geospatial data and achieved the data type conversion, thus finally realized the storage and analysis of these large-scale geographical data. The main research contents of this paper are as follows:1) Fully investigated the various concepts of cloud computing, summarized its main advantages and thoroughly studied two core underlying concepts of Hadoop: Distributed File System (HDFS) and MapReduce programming framework; 2) Built up MapReduce modal by combining with GIS Tools for Hadoop supported by Esri, we realized the purpose of efficient analyzing large-scale geospatial data. The research exhibits that geospatial data collecting and geospatial analysis can be well accomplished under cloud computing environment. Besides, the inefficiency of geospatial data management under traditional system can be sufficiently reduced. The methods and process used in this case may able to inspire new ideas for similar cases.