Abstract:Martian terrain segmentation plays a crucial role in autonomous navigation and safe driving of Mars rovers as well as global analysis of Martian geological landforms. However, most deep learning-based segmentation models cannot effectively handle the challenges of highly unstructured and unbalanced terrain distribution on the Martian surface, thus leading to inadequate adaptability and generalization ability. In this paper, we propose a novel multi-view Martian Terrain Segmentation framework (MTSNet) by developing an efficient Martian Terrain text-Guided Segment Anything Model (MTG-SAM) and combining it with a tailored Local Terrain Feature Enhancement Network (LTEN) to capture intricate terrain details. Specifically, the proposed MTG-SAM is equipped with a Terrain Context attention Adapter Module (TCAM) to efficiently and effectively unleashing the model adaptability and transferability on Mars-specific terrain distribution. Then, a Local Terrain Feature Enhancement Network (LTEN) is designated to compensate for the limitations of MTG-SAM in capturing the fine-grained local terrain features of Mars surface. Afterwards, a simple yet efficient Gated Fusion Module (GFM) is introduced to dynamically merge the global contextual features from MTG-SAM encoder and the local refined features from LTEN module for comprehensive terrain feature learning. Moreover, the proposed MTSNet enables terrain-specific text as prompts resolving the efficiency issue of existing methods that require costly annotation of bounding boxes or foreground points. Experimental results on AI4Mars and ConeQuest datasets demonstrate that our proposed MTSNet can effectively learns the unique Martian terrain feature distribution and achieves state-of-the-art performance on multi-view terrain segmentation from both the perspectives of the Mars rover and the satellite remote sensing. Code is available at https://github.com/raoxuefeng/mtsnet.

MTSNet: Joint Feature Adaptation and Enhancement for Text-Guided Multi-view Martian Terrain Segmentation

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

From One To Many: Unsupervised Traversable Area Segmentation In Off-Road Environment

Low-cost Mars Terrain Classification System Based on Coarse-grained Annotation

SegMarsViT: Lightweight Mars Terrain Segmentation Network for Autonomous Driving in Planetary Exploration

A terrain segmentation network for navigable areas with global strip reliability evaluation and dynamic fusion

Improved Mars Terrain Segmentation in Terms of Style Transfer and Network Architecture

MarsNet: Automated Rock Segmentation with Transformers for Tianwen-1 Mission

Deep Learning based Semantic Segmentation for Mars Rover Terrain Classification

MCMCNet: A Semi-supervised Road Extraction Network for High-resolution Remote Sensing Images Via Multiple Consistency and Multi-task Constraints

MarsMapNet: A Novel Superpixel-Guided Multiview Feature Fusion Network for Efficient Martian Landform Mapping

MarsSeg: Mars Surface Semantic Segmentation with Multi-level Extractor and Connector

A hybrid attention semantic segmentation network for unstructured terrain on Mars

MarsScapes and UDAFormer: A Panorama Dataset and a Transformer-Based Unsupervised Domain Adaptation Framework for Martian Terrain Segmentation

3D Semantic Terrain Reconstruction of Monocular Close-up Images of Martian Terrains

MarsFormer: Martian Rock Semantic Segmentation With Transformer

Mobile-DeepRFB: A Lightweight Terrain Classifier for Automatic Mars Rover Navigation

C-PLES: Contextual Progressive Layer Expansion with Self-attention for Multi-class Landslide Segmentation on Mars using Multimodal Satellite Imagery

Mixed-domain Training Improves Multi-Mission Terrain Segmentation

MTANet: Multitask-Aware Network with Hierarchical Multimodal Fusion for RGB-T Urban Scene Understanding

Light4Mars: A Lightweight Transformer Model for Semantic Segmentation on Unstructured Environment Like Mars