Abstract:Lossless data compression is a crucial and computing-intensive application in data-centric scenarios. To reduce the CPU overhead, FPGA-based accelerators have been proposed to offload compression workloads. However, most existing schemes have the problem of an imbalanced resource utilization and a poor practicability. In this paper, we propose HybriDC, an adaptive resource-efficient CPU-FPGA heterogeneous acceleration system for lossless data compression. Leveraging complementary advantages of the heterogeneous architecture, HybriDC provides a universal end-to-end compression acceleration framework with application compatibility and performance scalability. To optimize the hardware compression kernel design, we build a performance-resource model of the compression algorithm taking into account the design goal, compression performance, available resources, etc. According to the deduced resource-balanced design principle, the compression algorithm parameters are fine-tuned, which reduces 32% of the block RAM usage of the LZ4 kernel. In the parallel compression kernel implementation, a memory-efficient parallel hash table with an extra checksum is proposed, which supports parallel processing and improves the compression ratio without extra memory. We develop an LZ4-based HybriDC system prototype and evaluate it in detail. Our LZ4 compression kernel achieves state-of-the-art memory efficiency, 2.5-4× better than existing designs with comparable compression ratios. The evaluation of total resource utilization and end-to-end throughput demonstrates the excellent scalability of HybriDC. In power efficiency, the four-kernel HybriDC prototype achieves a threefold advantage over the standard LZ4 algorithm.

Data Compression and Storage under High Speed Network

Real-time Data Transmission of the Digital Simulation System Based on Network

Multi-core Lossless Compression System for High-speed Data and Its FPGA Implementation

Test Data Compression for System-on-Chip Using Advanced Frequency-Directed Run-Length(AFDR) Codes

A Hardware Implementation of Real Time Lossless Data Compression and Decompression Circuits

MetaZip

FPGA-Based High-Performance Data Compression Deep Neural Network Accelerator

A self-aware data compression system on FPGA in Hadoop

A Simple and Effective Data Compress Algorithm in Wireless Sensor Networks

Data Transmission in Mobile Edge Networks: Whether and Where to Compress?

HybriDC: A Resource-Efficient CPU-FPGA Heterogeneous Acceleration System for Lossless Data Compression

A Real-Time Image Row-Compression Method for High-Definition USB Cameras Based on FPGA

General Adaptive Lossless Compression for Multi-Channel Sensor Signals

FPGA Acceleration of Zstd Compression Algorithm

Parallel Data Compression Techniques

High-performance Effective Scientific Error-bounded Lossy Compression with Auto-tuned Multi-component Interpolation

Novel Data Compression Algorithm for Transmission Line Condition Monitoring

Real-time lossless image compression by dynamic Huffman coding hardware implementation

A Versatile Compression Method for Floating-Point Data Stream

Design and implementation of aerial remote sensing data compression system based on DSP

MetaZip: a high-throughput and efficient accelerator for DEFLATE