Abstract:3D volumetric image processing has attracted increasing attention in the last decades, in which one major research area is to develop efficient lossless volumetric image compression techniques to better store and transmit such images with massive amount of information. In this work, we propose the first end-to-end optimized learning framework for losslessly compressing 3D volumetric data. Our approach builds upon a hierarchical compression scheme by additionally introducing the intra-slice auxiliary features and estimating the entropy model based on both intra-slice and inter-slice latent priors. Specifically, we first extract the hierarchical intra-slice auxiliary features through multi-scale feature extraction modules. Then, an Intra-slice and Inter-slice Conditional Entropy Coding module is proposed to fuse the intra-slice and inter-slice information from different scales as the context information. Based on such context information, we can predict the distributions for both intra-slice auxiliary features and the slice images. To further improve the lossless compression performance, we also introduce two new gating mechanisms called Intra-Gate and Inter-Gate to generate the optimal feature representations for better information fusion. Eventually, we can produce the bitstream for losslessly compressing volumetric images based on the estimated entropy model. Different from the existing lossless volumetric image codecs, our end-to-end optimized framework jointly learns both intra-slice auxiliary features at different scales for each slice and inter-slice latent features from previously encoded slices for better entropy estimation. The extensive experimental results indicate that our framework outperforms the state-of-the-art hand-crafted lossless volumetric image codecs (e.g., JP3D) and the learning-based lossless image compression method on four volumetric image benchmarks for losslessly compressing both 3D Medical Images and Hyper-Spectral Images.

Total Variation Reduction for Lossless Compression of HPC Applications

AMP: Total Variation Reduction for Lossless Compression Via Approximate Median-based Preconditioning

Accelerating Lossy Compression on HPC Datasets Via Partitioning Computation for Parallel Processing

Performance Optimization for Relative-Error-Bounded Lossy Compression on Scientific Data.

Accelerating Relative-error Bounded Lossy Compression for HPC Datasets with Precomputation-Based Mechanisms.

High Performance Lossless Compression of Scientific Floating Data

ZFP-X: Efficient Embedded Coding for Accelerating Lossy Floating Point Compression.

A Near Lossless Compression Domain Volume Rendering Algorithm for Floating-Point Time-Varying Volume Data

FCBench: Cross-Domain Benchmarking of Lossless Compression for Floating-Point Data

HoSZp: An Efficient Homomorphic Error-bounded Lossy Compressor for Scientific Data

Use cases of lossy compression for floating-point data in scientific data sets

High-Ratio Lossy Compression: Exploring the Autoencoder to Compress Scientific Data

A Versatile Compression Method for Floating-Point Data Stream

High-performance Effective Scientific Error-bounded Lossy Compression with Auto-tuned Multi-component Interpolation

TAC+: Optimizing Error-Bounded Lossy Compression for 3D AMR Simulations

cuSZ-$i$: High-Ratio Scientific Lossy Compression on GPUs with Optimized Multi-Level Interpolation

Dynamic Quality Metric Oriented Error-bounded Lossy Compression for Scientific Datasets

Identifying Latent Reduced Models to Precondition Lossy Compression

Black-Box Statistical Prediction of Lossy Compression Ratios for Scientific Data

LCP: Enhancing Scientific Data Management with Lossy Compression for Particles

Exploiting Intra-Slice and Inter-Slice Redundancy for Learning-Based Lossless Volumetric Image Compression