Rapid Distributed Fine-tuning of a Segmentation Model Onboard Satellites

Meghan Plumridge,Rasmus Maråk,Chiara Ceccobello,Pablo Gómez,Gabriele Meoni,Filip Svoboda,Nicholas D. Lane

2024-11-27

Abstract:Segmentation of Earth observation (EO) satellite data is critical for natural hazard analysis and disaster response. However, processing EO data at ground stations introduces delays due to data transmission bottlenecks and communication windows. Using segmentation models capable of near-real-time data analysis onboard satellites can therefore improve response times. This study presents a proof-of-concept using MobileSAM, a lightweight, pre-trained segmentation model, onboard Unibap iX10-100 satellite hardware. We demonstrate the segmentation of water bodies from Sentinel-2 satellite imagery and integrate MobileSAM with PASEOS, an open-source Python module that simulates satellite operations. This integration allows us to evaluate MobileSAM's performance under simulated conditions of a satellite constellation. Our research investigates the potential of fine-tuning MobileSAM in a decentralised way onboard multiple satellites in rapid response to a disaster. Our findings show that MobileSAM can be rapidly fine-tuned and benefits from decentralised learning, considering the constraints imposed by the simulated orbital environment. We observe improvements in segmentation performance with minimal training data and fast fine-tuning when satellites frequently communicate model updates. This study contributes to the field of onboard AI by emphasising the benefits of decentralised learning and fine-tuning pre-trained models for rapid response scenarios. Our work builds on recent related research at a critical time; as extreme weather events increase in frequency and magnitude, rapid response with onboard data analysis is essential.

Machine Learning,Computer Vision and Pattern Recognition,Distributed, Parallel, and Cluster Computing

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the real - time processing delay of Earth Observation (EO) satellite data in disaster analysis and response. Specifically:

1. **Real - time data processing delay**: The traditional satellite data processing flow requires transmitting data from the satellite to the ground station for processing and then distributing it to end - users. This process causes a significant delay in real - time disaster response because the satellite must first reach a specific position before it can start transmitting data to the ground station. Moreover, with the increase in the number of Earth - observation satellites and the complication of satellite instruments, the amount of data has increased dramatically, while the existing bandwidth and communication windows are limited, which further exacerbates the data - transmission bottleneck.

2. **Requirement for rapid response**: As the frequency and intensity of extreme weather events keep increasing, rapid response to disaster events has become particularly important. Therefore, researchers have explored the possibility of directly running segmentation models on satellites to achieve near - real - time data analysis and thus shorten the response time.

3. **Adaptability of lightweight models**: Although deep - learning models perform excellently in processing Earth - observation data, they usually require a large amount of computing resources and training data. In order to run these models on resource - constrained satellite hardware, researchers have used lightweight pre - trained segmentation models (such as MobileSAM) and carried out rapid fine - tuning on the satellite to adapt to specific disaster scenarios.

By solving these problems, this research aims to improve the speed and efficiency of disaster response, especially in cases where data transmission is limited, enabling real - time or near - real - time disaster analysis and response.