Abstract:Studying the solar system and especially the Sun relies on the data gathered daily from space missions. These missions are data-intensive and compressing this data to make them efficiently transferable to the ground station is a twofold decision to make. Stronger compression methods, by distorting the data, can increase data throughput at the cost of accuracy which could affect scientific analysis of the data. On the other hand, preserving subtle details in the compressed data requires a high amount of data to be transferred, reducing the desired gains from compression. In this work, we propose a neural network-based lossy compression method to be used in NASA's data-intensive imagery missions. We chose NASA's SDO mission which transmits 1.4 terabytes of data each day as a proof of concept for the proposed algorithm. In this work, we propose an adversarially trained neural network, equipped with local and non-local attention modules to capture both the local and global structure of the image resulting in a better trade-off in rate-distortion (RD) compared to conventional hand-engineered codecs. The RD variational autoencoder used in this work is jointly trained with a channel-dependent entropy model as a shared prior between the analysis and synthesis transforms to make the entropy coding of the latent code more effective. Our neural image compression algorithm outperforms currently-in-use and state-of-the-art codecs such as JPEG and JPEG-2000 in terms of the RD performance when compressing extreme-ultraviolet (EUV) data. As a proof of concept for use of this algorithm in SDO data analysis, we have performed coronal hole (CH) detection using our compressed images, and generated consistent segmentations, even at a compression rate of $\sim0.1$ bits per pixel (compared to 8 bits per pixel on the original data) using EUV data from SDO.

Neural Compression of Atmospheric States

HiHa: Introducing Hierarchical Harmonic Decomposition to Implicit Neural Compression for Atmospheric Data

Compressing high-resolution data through latent representation encoding for downscaling large-scale AI weather forecast model

Compressive Representations of Weather Scenes for Strategic Air Traffic Flow Management

Neural-based Compression Scheme for Solar Image Data

Machine Learning Techniques for Data Reduction of Climate Applications

CRA5: Extreme Compression of ERA5 for Portable Global Climate and Weather Research via an Efficient Variational Transformer

Hierarchical Autoencoder-based Lossy Compression for Large-scale High-resolution Scientific Data

Neural NeRF Compression

Compression and Conditional Emulation of Climate Model Output

Hybrid Approaches for Data Reduction of Spatiotemporal Scientific Applications

ECNR: Efficient Compressive Neural Representation of Time-Varying Volumetric Datasets

Compressing Hyperspectral Images Into Multilayer Perceptrons Using Fast-Time Hyperspectral Neural Radiance Fields

Spatiotemporally adaptive compression for scientific dataset with feature preservation -- a case study on simulation data with extreme climate events analysis

Scientific data lossless compression using fast neural network

An Introduction to Neural Data Compression

Hyperspectral Image Compression Using Sampling and Implicit Neural Representations

Efficient Compression of Sparse Accelerator Data Using Implicit Neural Representations and Importance Sampling

Neural Image Compression Via Attentional Multi-scale Back Projection and Frequency Decomposition

Neural 3D Scene Compression via Model Compression

A priori compression of convolutional neural networks for wave simulators