Efficient onboard multi-task AI architecture based on self-supervised learning

Gabriele Inzerillo,Diego Valsesia,Enrico Magli

2024-08-19

Abstract:There is growing interest towards the use of AI directly onboard satellites for quick analysis and rapid response to critical events such as natural disasters. This paper presents a blueprint to the mission designer for the development of a modular and efficient deep learning payload to address multiple onboard inference tasks. In particular, we design a self-supervised lightweight backbone that provides features to efficient task-specific heads. The latter can be developed independently and with reduced data labeling requirements thanks to the frozen backbone. Experiments on three sample tasks of cloud segmentation, flood detection, and marine debris classification on a 7W embedded system show competitive results with inference quality close to high-complexity state-of-the-art models and high throughput in excess of 8 Mpx/s.

Image and Video Processing

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to implement an efficient multi - task AI architecture on satellites, so as to be able to analyze data in real - time and respond quickly to critical events, such as natural disasters. Specifically, the paper proposes a lightweight backbone network design based on self - supervised learning. This network can provide features to different task - specific heads, which can be developed independently and require less data - labeling requirements. Through experiments on three sample tasks (cloud segmentation, flood detection, and marine garbage classification) on a 7W embedded system, the paper shows that its method has the characteristics of high throughput and low power consumption while maintaining the inference quality close to that of high - complexity state - of - the - art models.

The main contributions of the paper are as follows:
1. **Lightweight Backbone Network**: A lightweight backbone network is designed, which can extract multi - resolution features from multispectral input images. These features are useful for tasks at multiple spatial resolutions.
2. **Task - Specific Heads**: The task - specific head design is proposed. These heads can be developed independently of the backbone network, reducing the need for a large amount of labeled data.
3. **Self - Supervised Learning**: The self - supervised learning method is used to train the backbone network, making use of a large number of unlabeled images to generate generalized feature representations.
4. **Modular Design**: The entire architecture is modular, and new task heads can be added without changing the weights of the backbone network, thus supporting multi - task parallel processing.
5. **Efficient Inference**: The memory requirements and inference speed of the model are further optimized through quantization techniques, enabling it to operate efficiently on embedded devices.

In conclusion, this paper aims to solve the efficiency and resource limitation problems of multi - task AI inference on satellites. By designing a lightweight, modular, and efficient AI architecture, it achieves high - quality multi - task processing capabilities under low - power - consumption conditions.