Mixed-domain Training Improves Multi-Mission Terrain Segmentation

Grace Vincent,Alice Yepremyan,Jingdao Chen,Edwin Goh

DOI: https://doi.org/10.48550/arXiv.2209.13674

2022-09-28

Abstract:Planetary rover missions must utilize machine learning-based perception to continue extra-terrestrial exploration with little to no human presence. Martian terrain segmentation has been critical for rover navigation and hazard avoidance to perform further exploratory tasks, e.g. soil sample collection and searching for organic compounds. Current Martian terrain segmentation models require a large amount of labeled data to achieve acceptable performance, and also require retraining for deployment across different domains, i.e. different rover missions, or different tasks, i.e. geological identification and navigation. This research proposes a semi-supervised learning approach that leverages unsupervised contrastive pretraining of a backbone for a multi-mission semantic segmentation for Martian surfaces. This model will expand upon the current Martian segmentation capabilities by being able to deploy across different Martian rover missions for terrain navigation, by utilizing a mixed-domain training set that ensures feature diversity. Evaluation results of using average pixel accuracy show that a semi-supervised mixed-domain approach improves accuracy compared to single domain training and supervised training by reaching an accuracy of 97% for the Mars Science Laboratory's Curiosity Rover and 79.6% for the Mars 2020 Perseverance Rover. Further, providing different weighting methods to loss functions improved the models correct predictions for minority or rare classes by over 30% using the recall metric compared to standard cross-entropy loss. These results can inform future multi-mission and multi-task semantic segmentation for rover missions in a data-efficient manner.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve

This paper aims to address several key issues faced in terrain segmentation for Mars exploration missions:

1. **High Data Annotation Demand**: Current Mars terrain segmentation models require a large amount of annotated data to achieve acceptable performance. This not only increases the time and cost of data preparation but also requires the involvement of experts such as geologists and rover operators for annotation work.

2. **Poor Cross-Domain Adaptability**: Existing terrain segmentation models need to be retrained when deployed across different missions or exploration tasks. For example, when transferring from one mission (such as the Mars Science Laboratory mission) to another (such as the Mars 2020 mission), the model's performance significantly drops.

3. **Class Imbalance**: There is a class imbalance issue in Mars terrain data, where the number of samples for certain classes is much smaller than for others. For instance, minority classes like large rocks and sand occupy a small proportion of the dataset but are crucial for tasks such as rover navigation.

### Solution

To address the above issues, the paper proposes a semi-supervised learning method to improve multi-task terrain segmentation through the following steps:

1. **Contrastive Pre-training**: Utilize a large number of unlabeled images for unsupervised contrastive pre-training to extract general features. This method can reduce the dependency on annotated data and improve the model's generalization ability.

2. **Mixed Domain Training**: Use datasets from different exploration missions for mixed domain training to ensure the model can adapt to terrain segmentation under different tasks and environments. This helps maintain high performance when the model is transferred between different missions.

3. **Loss Function Re-weighting**: Address the class imbalance issue by introducing different loss functions (such as inverse frequency loss and recall loss) to improve the prediction accuracy of minority classes.

### Experimental Results

Experimental results show that the semi-supervised mixed domain training method outperforms single-domain training and fully supervised training on multiple evaluation metrics. Specifically:

- **Pixel Accuracy**: The model achieved 97% pixel accuracy on the Mars Science Laboratory mission's "Curiosity" rover and 79.6% pixel accuracy on the Mars 2020 mission's "Perseverance" rover.
- **Minority Class Prediction**: By using different loss function weighting methods, the model's prediction accuracy for minority classes improved by over 30%.

These results indicate that the proposed method can achieve multi-task and multi-domain terrain segmentation with high data efficiency, providing strong support for future planetary exploration missions.