Abstract:Since Landsat-1 first started to deliver volumes of pixels in 1972, the volumes of archived data in remote sensing data centers have increased continuously. Due to various satellite orbit parameters and the specifications of different sensors, the storage formats, projections, spatial resolutions, and revisit periods of these archived data are vastly different. In addition, the remote sensing data received continuously by each data center arrives at a faster code rate; it is best to ingest and archive the newly received data to ensure users have access to the latest data retrieval and distribution services. Hence, an excellent data integration, organization, and management program is urgently needed. However, the multi-source, massive, heterogeneous, and distributed storage features of remote sensing data have not only caused difficulties for integration across distributed data center spatial infrastructures, but have also resulted in the current modes of data organization and management being unable meet the rapid retrieval and access requirements of users. Hence, this paper proposes an object-oriented data technology (OODT) and SolrCloud-based remote sensing data integration and management framework across a distributed data center spatial infrastructure. In this framework, all of the remote sensing metadata in the distributed sub-centers are transformed into the International Standardization Organization (ISO) 19115-based unified format, and then ingested and transferred to the main center by OODT components, continuously or at regular intervals. In the main data center, in order to improve the efficiency of massive data retrieval, we proposed a logical segmentation indexing (LSI) model-based data organization approach, and took SolrCloud to realize the distributed index and retrieval of massive metadata. Finally, a series of distributed data integration, retrieval, and comparative experiments showed that our proposed distributed data integration and management program is effective and promises superior results. Specifically, the LSI model-based data organization and the SolrCloud-based distributed indexing schema was able to effectively improve the efficiency of massive data retrieval.

An Efficient Organization Method for Large-Scale and Long Time-Series Remote Sensing Data in a Cloud Computing Environment

A distributed storage method of remote sensing data based on image blocks organization

An Improved Distributed Storage and Query for Remote Sensing Data.

An In-memory Data-Cube Aware Distributed Data Discovery Across Clouds for Remote Sensing Big Data

Research on Improved Method of Storage and Query of Large-Scale Remote Sensing Images.

Cloud Storage of Massive Remote Sensing Data Based on Distributed File System

Research on remote sensing image storage management and a fast visualization system based on cloud computing technology

An Efficient and Scalable Framework for Processing Remotely Sensed Big Data in Cloud Computing Environments

Big Data Integration in Remote Sensing Across a Distributed Metadata-Based Spatial Infrastructure

Efficient Retrieval of Massive Ocean Remote Sensing Images via a Cloud-Based Mean-Shift Algorithm

Big Earth Observation Data Integration in Remote Sensing Based on a Distributed Spatial Framework

A High-Dimensional Indexing Model for Multi-Source Remote Sensing Big Data

Research on Distributed Storage and Efficient Distribution Technology of High Resolution Optical Remote Sensing Data

A Study on Data Storage and Management for Massive Remote Sensing Data Based on Multi-level Grid Model

A Cloud Computing Solution for Big Imagery Data Analytics

An Adaptive Geographic Meshing and Coding Method for Remote Sensing Data

Organization and Management of Mass Remotely Sensed Data for Content-Based Retrieval

Spark-based Adaptive Mapreduce Data Processing Method for Remote Sensing Imagery

On-Demand Processing for Remote Sensing Big Data Analysis

Pipscloud: High Performance Cloud Computing for Remote Sensing Big Data Management and Processing

Distribution-Based Approach for Efficient Storage and Indexing of Massive Infrared Hyperspectral Sounding Data