Abstract:We previously presented Soil Property and Object Classification (SPOC), a machine learning-based terrain classi-fier for Mars rovers, for automatically segmenting rover images by its surface type such as sand and bedrock. This paper presents a number of practical improvements to pave the way for potential future onboard deployment. First, we achieved 97.0 % overall pixel accuracy, evaluated against the classification generated by human experts on images from Mars Science Lab-oratory (MSL) missions. The substantial increase in accuracy was primarily enabled by the sheer volume of data used for training; we created a new large-scale dataset of Martian terrain labels, namely AI4Mars, which contains more than 400k labels contributed by citizen scientists for ~50k images taken by the Mars Exploration Rovers (MER) and Mars Science Laboratory (MSL) rover. Second, we demonstrated that SPOC can quickly adapt to a new mission landed on a previously unseen site. Specifically, we pretrained a model with MER and MSL data from the AI4Mars dataset and then adapted to the Mars 2020 Rover (M2020) by feeding a small volume of data between Sol 0 and 157; the adapted model was tested on Sol 200–203 and resulted in 84.2 % overall pixel accuracy and 93.4 % reliability (recall) for detecting sand, the most concerning class for rover's traversability. Third, we found that pretraining can substan-tially mitigate the decline of accuracy over time. We showed that the performance of a SPOC model pretrained with the ImageNet dataset and then trained by MSL images only up to Sol 390 remains comparable to a model trained by images up to Sol 1689 on the test data after Sol 1689. Fourth, we reimplemented SPOC with a light-weight convolutional neural network (CNN), MobileNetV2, which typically runs within tens of milliseconds (ms) on mobile processors such as Qualcomm's Snapdragon. Finally, we released the AI4Mars dataset to the public to encourage open innovation.

Terrain Classification Using Mars Raw Images Based on Deep Learning Algorithms with Application to Wheeled Planetary Rovers

Visual Traversability Analysis for Micro Planetary Rover

Deep Learning based Semantic Segmentation for Mars Rover Terrain Classification

Automatic Data Processing for Space Robotics Machine Learning

You Only Crash Once: Improved Object Detection for Real-Time, Sim-to-Real Hazardous Terrain Detection and Classification for Autonomous Planetary Landings

Low-cost Mars Terrain Classification System Based on Coarse-grained Annotation

Mixed-domain Training Improves Multi-Mission Terrain Segmentation

Towards Safer Planetary Exploration: A Hybrid Architecture for Terrain Traversability Analysis in Mars Rovers

Deepening the Sense of Touch in Planetary Exploration with Geometric and Topological Deep Learning

Multi-mission Terrain Classifier for Safe Rover Navigation and Automated Science

Learning Surface Terrain Classifications from Ground Penetrating Radar

Pixel to Elevation: Learning to Predict Elevation Maps at Long Range using Images for Autonomous Offroad Navigation

Vibration-Based Terrain Identification for Planetary Exploration Robots Using Support Vector Machine

Predicting Terrain Mechanical Properties in Sight for Planetary Rovers with Semantic Clues

Mars Terrain Segmentation with Less Labels

Mapping "Brain Terrain" Regions on Mars using Deep Learning

Scene understanding for a high-mobility walking robot

Planetary Mapping Using Deep Learning: A Method to Evaluate Feature Identification Confidence Applied to Habitats in Mars-Analog Terrain

Deep Learning Traversability Estimator for Mobile Robots in Unstructured Environments

AI4MARS: A Dataset for Terrain-Aware Autonomous Driving on Mars

A Preliminary Study on Intelligent Perception of Extraterrestrial Body Surface Based on Deep Learning and Transfer Learning