Abstract:Active storage devices and in-storage computing are proposed and developed in recent years to effectively reduce the amount of required data traffic and to improve the overall application performance. They are especially preferred in the compute-storage disaggregated infrastructure. In both techniques, a simple computing module is added to storage devices/servers such that some stored data can be processed in the storage devices/servers before being transmitted to application servers. This can reduce the required network bandwidth and offload certain computing requirements from application servers to storage devices/servers. However, several challenges exist when designing an in-storage computing- based architecture for applications. These include what computing functions need to be offloaded, how to design the protocol between in-storage modules and application servers, and how to deal with the caching issue in application servers. HBase is an important and widely used distributed Key-Value Store. It stores and indexes key-value pairs in large files in a storage system like HDFS. However, its performance especially read performance, is impacted by the heavy traffics between HBase RegionServers and storage servers in the compute-storage disaggregated infrastructure when the available network bandwidth is limited. We propose an I n- S torage-based HBase architecture, called IS-HBase , to improve the overall performance and to address the aforementioned challenges. First, IS-HBase executes a data pre-processing module ( I n- S torage S can N er, called ISSN ) for some read queries and returns the requested key-value pairs to RegionServers instead of returning data blocks in HFile. IS-HBase carries out compactions in storage servers to reduce the large amount of data being transmitted through the network and thus the compaction execution time is effectively reduced. Second, a set of new protocols is proposed to address the communication and coordination between HBase RegionServers at computing nodes and ISSNs at storage nodes. Third, a new self-adaptive caching scheme is proposed to better serve the read queries with fewer I/O operations and less network traffic. According to our experiments, the IS-HBase can reduce up to 97% network traffic for read queries and the throughput (queries per second) is significantly less affected by the fluctuation of available network bandwidth. The execution time of compaction in IS-HBase is only about 6.31% – 41.84% of the execution time of legacy HBase. In general, IS-HBase demonstrates the potential of adopting in-storage computing for other data-intensive distributed applications to significantly improve performance in compute-storage disaggregated infrastructure.

Capability-aware Data Placement for Heterogeneous Active Storage Systems

A Holistic Heterogeneity-Aware Data Placement Scheme for Hybrid Parallel I/O Systems.

A cost-aware region-level data placement scheme for hybrid parallel I/O systems

DEAM：Decoupled, Expressive, Area-Efficient Metadata Cache

Performance-Aware Data Placement in Hybrid Parallel File Systems.

Cost-Aware Region-Level Data Placement in Multi-Tiered Parallel I/O Systems.

Skewed Data Distribution for Active Storage Systems on Hybrid Servers

A Migratory Heterogeneity-Aware Data Layout Scheme for Parallel File Systems.

hStorage-DB: Heterogeneity-aware Data Management to Exploit the Full Capability of Hybrid Storage Systems

Affinity and Conflict-Aware Placement of Virtual Machines in Heterogeneous Data Centers.

Application and Storage-Aware Data Placement and Job Scheduling for Hadoop Clusters.

A Heterogeneity-Aware Region-Level Data Layout for Hybrid Parallel File Systems.

HARL: Optimizing Parallel File Systems with Heterogeneity-Aware Region-Level Data Layout

Boosting Parallel File System Performance Via Heterogeneity-Aware Selective Data Layout.

A Data Placement Scheme with Time-Related Model for P2P Storages

HAS: Heterogeneity-Aware Selective Data Layout Scheme for Parallel File Systems on Hybrid Servers

Optimal Data Placement for Data-Sharing Scientific Workflows in Heterogeneous Edge-Cloud Computing Environments

An Analytical Model-based Capacity Planning Approach for Building CSD-based Storage Systems

An adaptive read/write optimized algorithm for Ceph heterogeneous systems via performance prediction and multi-attribute decision making

IS-HBase: an In-Storage Computing Optimized HBase with I/O Offloading and Self-Adaptive Caching in Compute-Storage Disaggregated Infrastructure

Analysis and Performance Study for Coordinated Hierarchical Cache Placement Strategies