Abstract:With the rapidly increasing number of satellites in space and their enhanced capabilities, the amount of earth observation images collected by satellites is exceeding the transmission limits of satellite-to-ground links. Although existing learned image compression solutions achieve remarkable performance by using a sophisticated encoder to extract fruitful features as compression and using a decoder to reconstruct, it is still hard to directly deploy those complex encoders on current satellites' embedded GPUs with limited computing capability and power supply to compress images in orbit. In this paper, we propose COSMIC, a simple yet effective learned compression solution to transmit satellite images. We first design a lightweight encoder (i.e. reducing FLOPs by $2.6\sim 5\times $) on satellite to achieve a high image compression ratio to save satellite-to-ground links. Then, for reconstructions on the ground, to deal with the feature extraction ability degradation due to simplifying encoders, we propose a diffusion-based model to compensate image details when decoding. Our insight is that satellite's earth observation photos are not just images but indeed multi-modal data with a nature of Text-to-Image pairing since they are collected with rich sensor data (e.g. coordinates, timestamp, etc.) that can be used as the condition for diffusion generation. Extensive experiments show that COSMIC outperforms state-of-the-art baselines on both perceptual and distortion metrics.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve With the rapid increase in the number of satellites in space and their enhanced capabilities, the volume of Earth observation images collected by satellites has exceeded the transmission limits between satellites and ground stations. Although existing learning-based image compression solutions extract rich features through complex encoders and reconstruct images during decoding, these complex encoders are difficult to deploy directly on the embedded GPUs of current satellites due to their limited computational power and power supply. This paper proposes a method called COSMIC (Compress Satellite Images Efficiently via Diffusion Compensation), which aims to efficiently compress satellite images to reduce the transmission burden. Specifically, COSMIC designs a lightweight encoder to achieve a high image compression ratio, thereby saving transmission bandwidth from the satellite to the ground. Meanwhile, to address the reduced feature extraction capability caused by the simplified encoder during ground decoding, COSMIC introduces a compensation method based on a diffusion model, utilizing the multimodal characteristics of satellite images (such as coordinates, timestamps, and other sensor data) to compensate for image details. ### Main Contributions 1. **Lightweight Encoder**: Designed a lightweight image compression encoder that reduces computational load (FLOPs), suitable for embedded GPUs on satellites. 2. **Diffusion Model Compensation**: Proposed a compensation method based on a stable diffusion model, utilizing the multimodal information of satellite images (such as sensor data) to compensate for image details. 3. **Detailed Analysis and Datasets**: Conducted a detailed analysis of the characteristics of satellite images and constructed two datasets for satellite image transmission scenarios, considering typical satellite image transmission tasks (such as stitching scenes). ### Experimental Results Experimental results show that COSMIC outperforms existing state-of-the-art baseline methods in terms of perceptual and distortion metrics. Particularly at low bit rates, COSMIC significantly improves the visual quality and structural similarity of images through effective compensation by the diffusion model. Additionally, COSMIC demonstrates better structural and color consistency in high-resolution image reconstruction, especially in stitched images, avoiding common misalignment and color difference issues seen in other methods. ### Conclusion COSMIC effectively addresses key issues in satellite image compression and transmission through a lightweight encoder and a diffusion model-based compensation method, providing new ideas and technical support for future satellite image processing.

COSMIC: Compress Satellite Images Efficiently via Diffusion Compensation

Spatial-spectral Encoding and Dictionary Optimization in Compressive Single-Pixel Hyperspectral Imaging Based on Mutual Coherence Minimization

Hybrid Compressive Spectral Imaging with Coded Aperture Optimization Based on Coherence Minimization

Convolutional Neural Network Based Computational Imaging Spectroscopy

Compressed learning based onboard semantic compression for remote sensing platforms

Compressing Repeated Content Within Large-Scale Remote Sensing Images

Compression Supports Spatial Deep Learning

Lightweight Bit-Depth Recovery Network for Gaofen Satellite Multispectral Image Compression

Earth+: on-board satellite imagery compression leveraging historical earth observations

High-throughput Onboard Hyperspectral Image Compression with Ground-based CNN Reconstruction

Data downlink prioritization using image classification on-board a 6U CubeSat

Multispectral image compression based on DSC combined with CCSDS-IDC

Full-Process Adaptive Encoding and Decoding Framework for Remote Sensing Images Based on Compression Sensing

Reconstruction Technique Based on the Theory of Compressed Sensing Satellite Images

DCSN: Deep Compressed Sensing Network for Efficient Hyperspectral Data Transmission of Miniaturized Satellite

Real-Time Compressed Sensing for Joint Hyperspectral Image Transmission and Restoration for CubeSat

Efficient On-Board Compression for Arbitrary-Shaped Cloud-Covered Remote Sensing Images via Adaptive Filling and Controllable Quantization

Dual-branch spectral–spatial feature extraction network for multispectral image compression

Content-aware Deep Perceptual Image Compression

Compression Ratio Learning and Semantic Communications for Video Imaging