Abstract:LiDAR odometry estimation and 3D semantic segmentation are crucial for autonomous driving, which has achieved remarkable advances recently. However, these tasks are challenging due to the imbalance of points in different semantic categories for 3D semantic segmentation and the influence of dynamic objects for LiDAR odometry estimation, which increases the importance of using representative/salient landmarks as reference points for robust feature learning. To address these challenges, we propose a saliency-guided approach that leverages attention information to improve the performance of LiDAR odometry estimation and semantic segmentation models. Unlike in the image domain, only a few studies have addressed point cloud saliency information due to the lack of annotated training data. To alleviate this, we first present a universal framework to transfer saliency distribution knowledge from color images to point clouds, and use this to construct a pseudo-saliency dataset (i.e. FordSaliency) for point clouds. Then, we adopt point cloud-based backbones to learn saliency distribution from pseudo-saliency labels, which is followed by our proposed SalLiDAR module. SalLiDAR is a saliency-guided 3D semantic segmentation model that integrates saliency information to improve segmentation performance. Finally, we introduce SalLONet, a self-supervised saliency-guided LiDAR odometry network that uses the semantic and saliency predictions of SalLiDAR to achieve better odometry estimation. Our extensive experiments on benchmark datasets demonstrate that the proposed SalLiDAR and SalLONet models achieve state-of-the-art performance against existing methods, highlighting the effectiveness of image-to-LiDAR saliency knowledge transfer. Source code will be available at <a class="link-external link-https" href="https://github.com/nevrez/SalLONet" rel="external noopener nofollow">this https URL</a>.

LASO: Language-guided Affordance Segmentation on 3D Object

3D-TAFS: A Training-free Framework for 3D Affordance Segmentation

SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model

PartAfford: Part-level Affordance Discovery from 3D Objects

Segment Any 3D Object with Language

Language-Conditioned Affordance-Pose Detection in 3D Point Clouds

Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusions

Better Call SAL: Towards Learning to Segment Anything in Lidar

PAVLM: Advancing Point Cloud based Affordance Understanding Via Vision-Language Model

O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning

LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Dataset

PerLA: Perceptive 3D Language Assistant

SegPoint: Segment Any Point Cloud via Large Language Model

LAC-Net: Linear-Fusion Attention-Guided Convolutional Network for Accurate Robotic Grasping Under the Occlusion

Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifiers

3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding

Learning 6-DoF Fine-grained Grasp Detection Based on Part Affordance Grounding

Interactive Object Segmentation in 3D Point Clouds

Multimodal 3D Reasoning Segmentation with Complex Scenes

Attention-Guided Lidar Segmentation and Odometry Using Image-to-Point Cloud Saliency Transfer

LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding