Abstract:Hyperspectral infrared atmospheric sounding data, characterized by their high vertical resolution, play a crucial role in capturing three-dimensional atmospheric spatial information. The hyperspectral infrared atmospheric detectors HIRAS/HIRAS-II, mounted on the FY3D/EF satellite, have established an initial global coverage network for atmospheric sounding. The collaborative observation approach involving multiple satellites will improve both the coverage and responsiveness of data acquisition, thereby enhancing the overall quality and reliability of the data. In response to the increasing number of channels, the rapid growth of data volume, and the specific requirements of multi-satellite joint observation applications with infrared hyperspectral sounding data, this paper introduces an efficient storage and indexing method for infrared hyperspectral sounding data within a distributed architecture for the first time. The proposed approach, built on the Kubernetes cloud platform, utilizes the Google S2 discrete grid spatial indexing algorithm to establish a grid-based hierarchical model for unified metadata-embedded documents. Additionally, it optimizes the rowkey design using the BPDS model, thereby enabling the distributed storage of data in HBase. The experimental results demonstrate that the query efficiency of the Google S2 grid-based embedded document model is superior to that of the traditional flat model, achieving a query time that is only 35.6% of the latter for a dataset of 5 million records. Additionally, this method exhibits better data distribution characteristics within the global grid compared to the H3 algorithm. Leveraging the BPDS model, the HBase distributed storage system adeptly balances the node load and counteracts the detrimental effects caused by the accumulation of time-series remote sensing images. This architecture significantly enhances both storage and query efficiency, thus laying a robust foundation for forthcoming distributed computing.

Distributed Storage and Query Method of Satellite Image Data Based on HBase

Massive Remote Sensing Image Data Management Based On Hbase And Geosot

HBaseSpatial: A Scalable Spatial Data Storage Based on HBase

Massive Image Data Management Using Hbase And Mapreduce

A distributed storage method of remote sensing data based on image blocks organization

Cloud Storage of Massive Remote Sensing Data Based on Distributed File System

Distribution-Based Approach for Efficient Storage and Indexing of Massive Infrared Hyperspectral Sounding Data

ESSG-Based Global Big Data Spatially Correlate and Its Share Service

Research on remote sensing image storage management and a fast visualization system based on cloud computing technology

Efficient Spatial Big Data Storage and Query in HBase.

The Design and Implementation of Geographic Information Storage System Based on the Cloud Platform.

An Efficient Organization Method for Large-Scale and Long Time-Series Remote Sensing Data in a Cloud Computing Environment

Distributed storage of network measurement data on HBase

The Framework of Remote Sensing Image Map Service on Hadoop

A Study on Data Storage and Management for Massive Remote Sensing Data Based on Multi-level Grid Model

An Management and Visualization System for Massive Remote Sensing Image: Research and Implementation.

Distributed Storage And Index Of Vector Spatial Data Based On Hbase

Data-Intensive Supercomputing in the Cloud: Global Analytics for Satellite Imagery

A Mechanism of Remote Sensing Image for Parallel Processing Base on Splitting Blocks

Study of Efficiently Storaging Images Data Based on SDE API

An improved distributed storage model of remote sensing images based on the HDFS and pyramid structure