Abstract:High-performance computing is progressively assuming a fundamental role in advancing scientific research and engineering domains. However, the ever-expanding scales of scientific simulations pose challenges for efficient data I/O and storage. The data compression technology has garnered significant attention as a solution to reduce data transmission and storage costs while enhancing performance. In particular, the BZIP2 lossless compression algorithm has been widely used due to its exceptional compression ratio, moderate compression speed, high reliability, and open-source nature. This paper focuses on the design and realization of a parallelized BZIP2 algorithm tailored for deployment on the New-Generation Sunway supercomputing platform. By leveraging the unique cache patterns of the New-Generation Sunway processor, we propose the highly tuned multi-threading and multi-node implementations of the BZIP2 applications for different scenarios. Moreover, we also propose the efficient BZIP2 libraries based on the management processing element and computing processing element which support the commonly used high-level (de)compression interfaces. The test results indicate that the our multi-threading implementation achieves maximum speedup of 23.09x$$ \times $$ (8.57x$$ \times $$) in decompression(compression) compared to the sequential implementation. Furthermore, the multi-node implementation achieves 50.81% (26.35%) parallel efficiency and peak performance of 16.6 GB/s (52.8 GB/s) for compression(decompression) when scaling up to 2048 processes. This paper focuses on the design and realization of a parallelized BZIP2 algorithm tailored for deployment on the New-Generation Sunway supercomputing platform. By leveraging the unique cache patterns of the New-Generation Sunway processor, we propose the highly tuned multi-threading and multi-node implementations of the BZIP2 applications for different scenarios. Moreover, we also propose the efficient BZIP2 libraries based on the management processing element and computing processing element which support the commonly used high-level (de)compression interfaces.image

ZIPS:Interactive Parallelization System

Parallel Computing Technology for Wide Area Power System Modeling

A parallel computing method for irregular work

Interactive Fortran 77 parallel system

Design, Implementation of the Parallel C Language Based on C/S Mode in Distributed Systems

Japs: an Automatic Parallelizing System Based on Java

Parallel Computing of Shared Memory Multiprocessors Based on JOMP

Design and Implementation of a User-interactiveParallel Programming

Parallel Power Flow Algorithm Based on MPI and P-Q Analysis Method

Parallel Computing for Water Quality Equation Based on Jppf

J-Parallelio -- automatic parallelization framework for Java virtual machine code

Source-code-level Transformation and APT-Driven Parallelism Pre-processes for Embedded System Automated Design

A Numerical Model Oriented Large-scale Parallel I/O Optimization Method.

Distributed Parallel Optimization Computation Environment Based on Matlab

Author Response for "refactoring BZIP2 on the New-Generation Sunway Supercomputer"

Design and Implementation of Task Scheduling Protocol in JAPS

A Mechanism of Remote Sensing Image for Parallel Processing Base on Splitting Blocks

Parallel Visualization for Large-Scale Datasets

Communication performance analyze in large scale parallel program

A Path-Based Method of Parallelizing C++ Programs

Parallel Implementation for Dynamic Mesh Optimization on Distributed Computer System.