Starting with data: advancing spatial data science by building and sharing high-quality datasets

Yingjie Hu

DOI: https://doi.org/10.48550/arXiv.2007.08087

2020-07-16

Computers and Society

Abstract:Spatial data science has emerged in recent years as an interdisciplinary field. This position paper discusses the importance of building and sharing high-quality datasets for spatial data science.

What problem does this paper attempt to address?

Spatial data quality and beyond

Deren Li,Jingxiong Zhang,Huayi Wu

DOI: https://doi.org/10.1080/13658816.2012.719625

2012-12-01

International Journal of Geographical Information Science

Abstract:Issues of accuracy, uncertainty, and spatial data quality have been on the top of most GIScience research agendas around the world from the late 1980s. Ever since then, growing research efforts have been directed toward uncertainty characterization in spatial information, analysis, and applications, aiming for better understanding of spatial uncertainty and thus improved methods and techniques for assessing and managing data quality. Impressive progress has been made in various issues concerning data quality. In addition, growing research on extensions to the conventional norms of data quality, such as the quality aspects of geospatial information services, has been observed. Chinese researchers have contributed to this great cause by keeping abreast with the developments abroad and striving for their own innovative work. This paper reviews the past research on data quality-related issues and provides a perspective on future developments. These will be seen not only in continued research on theoretical and technical issues concerning data quality, but also in developments of tools for quality assessment and decision-making under uncertainty through geospatial information processing and applications.

geography, physical,computer science, information systems,information science & library science
A New Design of High-Performance Large-Scale GIS Computing at a Finer Spatial Granularity: A Case Study of Spatial Join with Spark for Sustainability

Feng Zhang,Jingwei Zhou,Renyi Liu,Zhenhong Du,Xinyue Ye

DOI: https://doi.org/10.3390/su8090926

IF: 3.9

2016-01-01

Sustainability

Abstract:Sustainability research faces many challenges as respective environmental, urban and regional contexts are experiencing rapid changes at an unprecedented spatial granularity level, which involves growing massive data and the need for spatial relationship detection at a faster pace. Spatial join is a fundamental method for making data more informative with respect to spatial relations. The dramatic growth of data volumes has led to increased focus on high-performance large-scale spatial join. In this paper, we present Spatial Join with Spark (SJS), a proposed high-performance algorithm, that uses a simple, but efficient, uniform spatial grid to partition datasets and joins the partitions with the built-in join transformation of Spark. SJS utilizes the distributed in-memory iterative computation of Spark, then introduces a calculation-evaluating model and in-memory spatial repartition technology, which optimize the initial partition by evaluating the calculation amount of local join algorithms without any disk access. We compare four in-memory spatial join algorithms in SJS for further performance improvement. Based on extensive experiments with real-world data, we conclude that SJS outperforms the Spark and MapReduce implementations of earlier spatial join approaches. This study demonstrates that it is promising to leverage high-performance computing for large-scale spatial join analysis. The availability of large-sized geo-referenced datasets along with the high-performance computing technology can raise great opportunities for sustainability research on whether and how these new trends in data and technology can be utilized to help detect the associated trends and patterns in the human-environment dynamics.
Opening practice: supporting reproducibility and critical spatial data science

Chris Brunsdon,Alexis Comber

DOI: https://doi.org/10.1007/s10109-020-00334-2

2020-08-08

Journal of Geographical Systems

Abstract:Abstract This paper reflects on a number of trends towards a more open and reproducible approach to geographic and spatial data science over recent years. In particular, it considers trends towards Big Data, and the impacts this is having on spatial data analysis and modelling. It identifies a turn in academia towards coding as a core analytic tool, and away from proprietary software tools offering ‘black boxes’ where the internal workings of the analysis are not revealed. It is argued that this closed form software is problematic and considers a number of ways in which issues identified in spatial data analysis (such as the MAUP) could be overlooked when working with closed tools, leading to problems of interpretation and possibly inappropriate actions and policies based on these. In addition, this paper considers the role that reproducible and open spatial science may play in such an approach, taking into account the issues raised. It highlights the dangers of failing to account for the geographical properties of data, now that all data are spatial (they are collected some where ), the problems of a desire for $$n$$ n = all observations in data science and it identifies the need for a critical approach. This is one in which openness, transparency, sharing and reproducibility provide a mantra for defensible and robust spatial data science.

geography
Spatial Data Science: Closing the human-spatial computing-environment loop

Benjamin Adams

DOI: https://doi.org/10.48550/arXiv.1910.06484

2019-10-15

Social and Information Networks

Abstract:Over the last decade, the term spatial computing has grown to have two different, though not entirely unrelated, definitions. The first definition of spatial computing stems from industry, where it refers primarily to new kinds of augmented, virtual, mixed-reality, and natural user interface technologies. A second definition coming out of academia takes a broader perspective that includes active research in geographic information science as well as the aforementioned novel UI technologies. Both senses reflect an ongoing shift toward increased interaction with computing interfaces and sensors embedded in the environment and how the use of these technologies influence how we behave and make sense of and even change the world we live in. Regardless of the definition, research in spatial computing is humming along nicely without the need to identify new research agendas or new labels for communities of researchers. However, as a field of research, it could be helpful to view spatial data science as the glue that coheres spatial computing with problem-solving and learning in the real world into a more holistic discipline.
SpatialData: an open and universal data framework for spatial omics

Luca Marconato,Giovanni Palla,Kevin A. Yamauchi,Isaac Virshup,Elyas Heidari,Tim Treis,Wouter-Michiel Vierdag,Marcella Toth,Sonja Stockhaus,Rahul B. Shrestha,Benjamin Rombaut,Lotte Pollaris,Laurens Lehner,Harald Vöhringer,Ilia Kats,Yvan Saeys,Sinem K. Saka,Wolfgang Huber,Moritz Gerstung,Josh Moore,Fabian J. Theis,Oliver Stegle

DOI: https://doi.org/10.1038/s41592-024-02212-x

IF: 48

2024-03-21

Nature Methods

Abstract:Spatially resolved omics technologies are transforming our understanding of biological tissues. However, the handling of uni- and multimodal spatial omics datasets remains a challenge owing to large data volumes, heterogeneity of data types and the lack of flexible, spatially aware data structures. Here we introduce SpatialData, a framework that establishes a unified and extensible multiplatform file-format, lazy representation of larger-than-memory data, transformations and alignment to common coordinate systems. SpatialData facilitates spatial annotations and cross-modal aggregation and analysis, the utility of which is illustrated in the context of multiple vignettes, including integrative analysis on a multimodal Xenium and Visium breast cancer study.

biochemical research methods
Big Issues for Big Data: challenges for critical spatial data analytics

Chris Brunsdon,Alexis Comber

DOI: https://doi.org/10.48550/arXiv.2007.11281

2020-08-11

Abstract:In this paper we consider some of the issues of working with big data and big spatial data and highlight the need for an open and critical framework. We focus on a set of challenges underlying the collection and analysis of big data. In particular, we consider 1) the issues related to inference when working with usually biased big data, challenging the assumed inferential superiority of data with observations, n, approaching N, the population (n->N), and the need for data science analysis that answer questions of practical significance or with greater emphasis n the size of the effect, rather than the truth or falsehood of a statistical statement; 2) the need to accept messiness in your data and to document all operations undertaken on the data because of this support of openness and reproducibility paradigms; and 3) the need to explicitly seek to understand the causes of bias, messiness etc in the data and the inferential consequences of using such data in analyses, by adopting critical approaches to spatial data science. In particular we consider the need to place individual data science studies in a wider social and economic contexts, along the the role of inferential theory in the presence of big data, and issues relating to messiness and complexity in big data.

Computers and Society
Geospatial big data for urban planning and urban management

Huayi Wu,Zhipeng Gui,Zelong Yang

DOI: https://doi.org/10.1080/10095020.2020.1854981

2020-10-01

Abstract:The recent ten years witnessed the great achievements on rich applications of Geospatial Big Data across a variety of disciplines. For example, a huge number of Landsat images are utilized in mapping high-resolution global forest cover and the global forest changes in the twenty-first century are explored (Hansen et al. <a href="#">2013</a>), which is impossible without the support of geospatial big data and the related automatic processing techniques. Based on the huge enterprise registration data in China, the economic and social development situations and trends are revealed by the non-statistic data and novel approaches (Li et al. <a href="#">2018</a>). City-wide fine-grained urban population distribution at building level is achieved by integrating and fusing multisource geospatial big data (Yao et al. <a href="#">2017</a>), which is usually not desired in traditional research. Geospatial Big Data provides a new transforming paradigm of scientific research especially at the crossroads of broad disciplines, including but not limited to the humanities, the physical sciences, engineering, and so on.

remote sensing
HBaseSpatial: A Scalable Spatial Data Storage Based on HBase

Ningyu Zhang,Guozhou Zheng,Huajun Chen,Jiaoyan Chen,Xi Chen

DOI: https://doi.org/10.1109/TrustCom.2014.83

2014-01-01

Abstract:Recent years, the scale of spatial data is developing more and more huge and its storage has encountered a lot of problems. Traditional DBMS can efficiently handle some big spatial data. However, popular open source relational database systems are overwhelmed by the high insertion rates, querying requirements and terabytes of data that these systems can handle. On the other hand, key-value storage can effectively support large scale operations. To resolve the problems of big vector spatial data's storage and query, we bring forward HBase Spatial, a scalable spatial dada storage based on HBase. At first, we analyze the distributed storage model of HBase. Then, we design a distributed storage and index model. Finally, the advantages of our storage model and index algorithm are proven by experiments with both big sample sets and typical benchmarks on cluster compared with MongoDB and Mysql, which shows that our model can effectively enhance the query speed of big spatial data and provide a good solution for storage.
Spatial Big Data Science

Zhe Jiang,Shashi Shekhar

DOI: https://doi.org/10.1007/978-3-319-60195-3

2017-01-01

Abstract:Emerging Spatial Big Data (SBD) has transformative potential in solving many grand societal challenges such as water resource management, food security, disaster response, and transportation. However, significant computational challenges exist in analyzing SBD due to the unique spatial characteristics including spatial autocorrelation, anisotropy, heterogeneity, multiple scales and resolutions which is illustrated in this book. This book also discusses current techniques for, spatial big data science with a particular focus on classification techniques for earth observation imagery big data. Specifically, the authors introduce several recent spatial classification techniques, such as spatial decision trees and spatial ensemble learning. Several potential future research directions are also discussed. This book targets an interdisciplinary audience including computer scientists, practitioners and researchers working in the field of data mining, big data, as well as domain scientists working in earth science (e.g., hydrology, disaster), public safety and public health. Advanced level students in computer science will also find this book useful as a reference.
Replicable and extensible spatial data acquisition

Matthew Haffner

DOI: https://doi.org/10.1080/15230406.2024.2409208

IF: 2.354

2024-10-20

Cartography and Geographic Information Science

Abstract:Having the ability to reproduce empirical results is foundational to the scientific process. Increasingly, there is an emphasis within the research community to create workflows which are not only reproducible but also replicable in that analytical approaches can be applied to new data. However, a disproportionate number of technical solutions designed to foster reproducibility and replicability have focused on data analysis, particularly within geography. This paper argues for greater attention on a phase of the research process which precedes analysis and is often taken for granted – that of data acquisition. Through code examples using the R Project for Statistical Computing, this paper demonstrates a path toward replicable spatial data acquisition for both secondary and primary data acquisition, with a focus on crowdsourced geographic information. The modular approach demonstrated is flexible in that it allows for straightforward replication but also enables extension to new spatial questions. Such work is valuable because it conserves labor, promotes research provenance, and allows for more robust analyses.

geography
Spatial Humanities and Geo-computation for Social Sciences:Advances and Applications

Kun QIN,Hui LIN,Yang YUE,Feng ZHANG,Jianya GONG

DOI: https://doi.org/10.11947/j.jggs.2022.0201

2022-01-01

Abstract:Humanities and Social Sciences ( HSS) are undergoing the transformation of spatialization and quantification. Geo-computation, with geoinformatics ( including RS: Remote Sensing;GIS: Geographical Information System;GNSS: Global Navigation Satellite System) , provides effective computational and spatialization methods and tools for HSS. Spatial Humanities and Geo-computation for Social Sciences ( SH&GSS) is a field coupling geo-computation, and geoinformatics, with HSS. This special issue accepted a set of contributions highlighting recent advances in methodologies and applications of SH&GSS, which are related to sentiment spatial analysis from social media data, emotional change spatial analysis from news data, spatial analysis of social media related to COVID-19, crime spatiotemporal analysis,"double evaluation"for Land Use/Land Cover (LUCC), Specially Protected Natural Areas (SPNA) analysis, editing behavior analysis of Volunteered Geographic Information (VGI), electricity consumption anomaly detection, First and Last Mile Problem ( FLMP ) of public transport, and spatial interaction network analysis for crude oil trade network. Based on these related researches, we aim to present an overview of SH&GSS, and propose some future research directions for SH&HSS.
Considerations on Geospatial Big Data

Zhen Liu,Huadong Guo,Changlin Wang

DOI: https://doi.org/10.1088/1755-1315/46/1/012058

2016-01-01

IOP Conference Series Earth and Environmental Science

Abstract:Geospatial data, as a significant portion of big data, has recently gained the full attention of researchers. However, few researchers focus on the evolution of geospatial data and its scientific research methodologies. When entering into the big data era, fully understanding the changing research paradigm associated with geospatial data will definitely benefit future research on big data. In this paper, we look deep into these issues by examining the components and features of geospatial big data, reviewing relevant scientific research methodologies, and examining the evolving pattern of geospatial data in the scope of the four ‘science paradigms’. This paper proposes that geospatial big data has significantly shifted the scientific research methodology from ‘hypothesis to data’ to ‘data to questions’ and it is important to explore the generality of growing geospatial data ‘from bottom to top’. Particularly, four research areas that mostly reflect data-driven geospatial research are proposed: spatial correlation, spatial analytics, spatial visualization, and scientific knowledge discovery. It is also pointed out that privacy and quality issues of geospatial data may require more attention in the future. Also, some challenges and thoughts are raised for future discussion.
Incorporating Multi-source Big Geo-data to Sense Spatial Heterogeneity Patterns in an Urban Space

LIU Yu,ZHAN Zhaohui,ZHU Di,CHAI Yanwei,MA Xiujun,WU Lun

DOI: https://doi.org/10.13203/j.whugis20170383

2018-01-01

Abstract:Multi-source big geo-data provides us an unprecedented opportunity to investigate geographic phenomena from perspective of their spatial distribution patterns,spatial interactions and dynamic evolution.Cities are the most concentrated areas of human activities and thus massive amount of geographic big data have been produced to improve our understanding of urban spaces.The spatial heterogeneity patterns in cities is an essential topic in geographic research and urban planning.Social sensing offers an analytical framework to characterize urban spatial heterogeneity from four dimensions:human,environment,statics and dynamics.This paper summarizes the contributions of different types of big geo-data in characterizing urban features.Borrowing the concept of "niche model" from ecological studies,a case study is introduced to demonstrate the quantification of spatial heterogeneity patterns in urban space incorporating multi-source big geo-data.Theoretical issues such as unit selection are also discussed to address some related problems.
Building benchmarking frameworks for supporting replicability and reproducibility: spatial and textual analysis as an example

Yingjie Hu

DOI: https://doi.org/10.48550/arXiv.2007.01978

2020-07-04

Abstract:Replicability and reproducibility (R&R) are critical for the long-term prosperity of a scientific discipline. In GIScience, researchers have discussed R&R related to different research topics and problems, such as local spatial statistics, digital earth, and metadata (Fotheringham, 2009; Goodchild, 2012; Anselin et al., 2014). This position paper proposes to further support R&R by building benchmarking frameworks in order to facilitate the replication of previous research for effective and effcient comparisons of methods and software tools developed for addressing the same or similar problems. Particularly, this paper will use geoparsing, an important research problem in spatial and textual analysis, as an example to explain the values of such benchmarking frameworks.

Information Retrieval
The Right Tools for the Job: The Case for Spatial Science Tool-Building

Geoff Boeing

DOI: https://doi.org/10.1111/tgis.12678

2020-08-13

Abstract:This paper was presented as the 8th annual Transactions in GIS plenary address at the American Association of Geographers annual meeting in Washington, DC. The spatial sciences have recently seen growing calls for more accessible software and tools that better embody geographic science and theory. Urban spatial network science offers one clear opportunity: from multiple perspectives, tools to model and analyze nonplanar urban spatial networks have traditionally been inaccessible, atheoretical, or otherwise limiting. This paper reflects on this state of the field. Then it discusses the motivation, experience, and outcomes of developing OSMnx, a tool intended to help address this. Next it reviews this tool's use in the recent multidisciplinary spatial network science literature to highlight upstream and downstream benefits of open-source software development. Tool-building is an essential but poorly incentivized component of academic geography and social science more broadly. To conduct better science, we need to build better tools. The paper concludes with paths forward, emphasizing open-source software and reusable computational data science beyond mere reproducibility and replicability.

Computers and Society,Physics and Society,Applications
Historical Spatial-Data Infrastructures for Archaeology: Towards a Spatiotemporal Big-Data Approach to Studying the Postindustrial City

Dan Trepal,Don Lafreniere,Jason Gilliland

DOI: https://doi.org/10.1007/s41636-020-00245-5

2020-05-28

Historical Archaeology

Abstract:While the use of geographic information systems (GIS) has become commonplace within the discipline of archaeology, the potential of a big-data approach to GIS is yet to be fully exploited within historical archaeology. Archaeologists inspired by developments in the social sciences and humanities have recently called for new ways of conceptualizing GIS as a process that is more theoretically satisfying and methodologically effective in its applications to archaeology. We respond to these calls by proposing a new approach for GIS in historical archaeology, an historical spatial-data infrastructure (HSDI). We outline the progression from historical GIS to the construction of an HSDI and present a series of case studies that demonstrate how using a spatiotemporal big-data-based approach expands the scale of archaeological inquiry to studying the postindustrial city.
Big spatial data for urban and environmental sustainability

Bo Huang,Jionghua Wang

DOI: https://doi.org/10.1080/10095020.2020.1754138

2020-04-02

Abstract:Eighty percent of big data are associated with spatial information, and thus are Big Spatial Data (BSD). BSD provides new and great opportunities to rework problems in urban and environmental sustainability with advanced BSD analytics. To fully leverage the advantages of BSD, it is integrated with conventional data (e.g. remote sensing images) and improved methods are developed. This paper introduces four case studies: (1) Detection of polycentric urban structures; (2) Evaluation of urban vibrancy; (3) Estimation of population exposure to PM2.5; and (4) Urban land-use classification via deep learning. The results provide evidence that integrated methods can harness the advantages of both traditional data and BSD. Meanwhile, they can also improve the effectiveness of big data itself. Finally, this study makes three key recommendations for the development of BSD with regards to data fusion, data and predicting analytics, and theoretical modeling.

remote sensing
Research on the Spatial Data Intelligent Foundation Model

Shaohua Wang,Xing Xie,Yong Li,Danhuai Guo,Zhi Cai,Yu Liu,Yang Yue,Xiao Pan,Feng Lu,Huayi Wu,Zhipeng Gui,Zhiming Ding,Bolong Zheng,Fuzheng Zhang,Jingyuan Wang,Zhengchao Chen,Hao Lu,Jiayi Li,Peng Yue,Wenhao Yu,Yao Yao,Leilei Sun,Yong Zhang,Longbiao Chen,Xiaoping Du,Xiang Li,Xueying Zhang,Kun Qin,Zhaoya Gong,Weihua Dong,Xiaofeng Meng

2024-08-28

Abstract:This report focuses on spatial data intelligent large models, delving into the principles, methods, and cutting-edge applications of these models. It provides an in-depth discussion on the definition, development history, current status, and trends of spatial data intelligent large models, as well as the challenges they face. The report systematically elucidates the key technologies of spatial data intelligent large models and their applications in urban environments, aerospace remote sensing, geography, transportation, and other scenarios. Additionally, it summarizes the latest application cases of spatial data intelligent large models in themes such as urban development, multimodal systems, remote sensing, smart transportation, and resource environments. Finally, the report concludes with an overview and outlook on the development prospects of spatial data intelligent large models.

Artificial Intelligence,Computer Vision and Pattern Recognition,Machine Learning
Improving data access democratizes and diversifies science

Abhishek Nagaraj,Esther Shears,Mathijs de Vaan

DOI: https://doi.org/10.1073/pnas.2001682117

IF: 11.1

2020-09-08

Proceedings of the National Academy of Sciences

Abstract:Significance Data access is critical to empirical research, but past work on open access is largely restricted to the life sciences and has not directly analyzed the impact of data access restrictions. We analyze the impact of improved data access on the quantity, quality, and diversity of scientific research. We focus on the effects of a shift in the accessibility of satellite imagery data from Landsat, a NASA program that provides valuable remote-sensing data. Our results suggest that improved access to scientific data can lead to a large increase in the quantity and quality of scientific research. Further, better data access disproportionately enables the entry of scientists with fewer resources, and it promotes diversity of scientific research.
Towards Data Science

yangyong zhu,yun xiong

DOI: https://doi.org/10.5334/dsj-2015-008

2015-01-01

Abstract:Currently, a huge amount of data is being rapidly generated in cyberspace. Datanature (all data in cyberspace) is forming due to a data explosion. Exploring the patterns and rules in datanature is necessary but difficult. A new discipline called Data Science is coming. It provides a type of novel research method (a data-intensive method) for natural and social sciences and goes beyond computer science in researching data. This paper presents the challenges presented by data and discusses what differentiates data science from the established sciences, data technologies, and big data. Our goal is to encourage data related researchers to transfer their focus towards this new science.

Starting with data: advancing spatial data science by building and sharing high-quality datasets

Spatial data quality and beyond

A New Design of High-Performance Large-Scale GIS Computing at a Finer Spatial Granularity: A Case Study of Spatial Join with Spark for Sustainability

Opening practice: supporting reproducibility and critical spatial data science

Spatial Data Science: Closing the human-spatial computing-environment loop

SpatialData: an open and universal data framework for spatial omics

Big Issues for Big Data: challenges for critical spatial data analytics

Geospatial big data for urban planning and urban management

HBaseSpatial: A Scalable Spatial Data Storage Based on HBase

Spatial Big Data Science

Replicable and extensible spatial data acquisition

Spatial Humanities and Geo-computation for Social Sciences:Advances and Applications

Considerations on Geospatial Big Data

Incorporating Multi-source Big Geo-data to Sense Spatial Heterogeneity Patterns in an Urban Space

Building benchmarking frameworks for supporting replicability and reproducibility: spatial and textual analysis as an example

The Right Tools for the Job: The Case for Spatial Science Tool-Building

Historical Spatial-Data Infrastructures for Archaeology: Towards a Spatiotemporal Big-Data Approach to Studying the Postindustrial City

Big spatial data for urban and environmental sustainability

Research on the Spatial Data Intelligent Foundation Model

Improving data access democratizes and diversifies science

Towards Data Science