Research on the Analysis and Statistic of Geographical Conditions Based on the Strategy of "grid Index + MapReduce"
LIN Yaping,DU Zhenhong,ZHANG Feng,LIU Renyi
DOI: https://doi.org/10.3785/j.issn.1008-9497.2017.06.004
2017-01-01
Abstract:The statistic of geographical conditions is the primary premise for the deep excavation and application of geographical data . However , the traditional centralized data storage and processing method based on a single computer are time-consuming ,inefficient and even unsupported .This paper creates a strategy called "Grid Index +MapReduce" to solve these problems .Firstly ,we design a blocking file organization and distributed storage mode of the census data of geographical situation based on the regular square grid ,and then make a double layer filtering method which combines the grid index and the accurate analysis .Lastly ,we build a parallel processing algorithm of statistic of the geography conditions based on MapReduce . The test results of performance comparison of the strategy of "Grid Index + MapReduce",the indexless MapReduce and ArcGIS software show that the method of"Grid Index + MapReduce" is much more efficient than the ArcGIS software ,and also has obvious efficiency advantages for the indexless MapReduce method . The study tries to provide an optimal scheme for the high-performance ,multi-type and high-volume statistic and analysis method for the data of geographical condition survey .