Abstract:High-performance computing is progressively assuming a fundamental role in advancing scientific research and engineering domains. However, the ever-expanding scales of scientific simulations pose challenges for efficient data I/O and storage. The data compression technology has garnered significant attention as a solution to reduce data transmission and storage costs while enhancing performance. In particular, the BZIP2 lossless compression algorithm has been widely used due to its exceptional compression ratio, moderate compression speed, high reliability, and open-source nature. This paper focuses on the design and realization of a parallelized BZIP2 algorithm tailored for deployment on the New-Generation Sunway supercomputing platform. By leveraging the unique cache patterns of the New-Generation Sunway processor, we propose the highly tuned multi-threading and multi-node implementations of the BZIP2 applications for different scenarios. Moreover, we also propose the efficient BZIP2 libraries based on the management processing element and computing processing element which support the commonly used high-level (de)compression interfaces. The test results indicate that the our multi-threading implementation achieves maximum speedup of 23.09x$$ \times $$ (8.57x$$ \times $$) in decompression(compression) compared to the sequential implementation. Furthermore, the multi-node implementation achieves 50.81% (26.35%) parallel efficiency and peak performance of 16.6 GB/s (52.8 GB/s) for compression(decompression) when scaling up to 2048 processes. This paper focuses on the design and realization of a parallelized BZIP2 algorithm tailored for deployment on the New-Generation Sunway supercomputing platform. By leveraging the unique cache patterns of the New-Generation Sunway processor, we propose the highly tuned multi-threading and multi-node implementations of the BZIP2 applications for different scenarios. Moreover, we also propose the efficient BZIP2 libraries based on the management processing element and computing processing element which support the commonly used high-level (de)compression interfaces.image

Closed Cubing on PC Clusters

Requirement-Based Data Cube Schema Design

C-Cubing: Efficient Computation of Closed Cubes by Aggregation-Based Checking

SCANCHUNK:AN EFFICIENT ALGORITHM FOR HUNTING DENSE REGIONS IN DATA CUBE

Towards the Building of a Dense-Region-based OLAP System

A Clustered Dwarf Structure to Speed Up Queries on Data Cubes

Parallel Simplification of Large Meshes on PC Clusters.

CubeExplorer: online exploration of data cubes.

CLUSTER-BASED OCEAN REMOTE SENSING IMAGE FUSION PARALLEL COMPUTING STRATEGY

Data Cube Structure Based on Operation of Sets

A Mechanism of Remote Sensing Image for Parallel Processing Base on Splitting Blocks

Regression Cubes with Lossless Compression and Aggregation

APIC: An Efficient Algorithm for Computing Iceberg Datacubes

Effective Data Stream Cube Structure

Dynamic construction of user defined virtual cubes

A Parallel Varied Density-Based Clustering Algorithm with Optimized Data Partition

Constructing Cube Blocks Effectively for Stream Data Analysis

3-D Partitioning for Large-Scale Graph Processing.

Author Response for "refactoring BZIP2 on the New-Generation Sunway Supercomputer"

Holistic Cube Analysis: A Query Framework for Data Insights

DESIGN AND IMPLEMENTATION OF POLYNOMIAL-PRECONDITIONED EBE-PCG PARALLEL ALGORITHM IN CLUSTER