Abstract:Light Detection and Ranging (LiDAR) and camera are two commonly used sensors to acquire data of different modalities in environmental perception. For autonomous vehicles operating in unstructured scenes, it is particularly important to fuse these two different data for semantic segmentation. Most existing methods rely solely on data from a single sensor, which partially limits segmentation performance. Some fusion-based methods, however, are unable to trade off the contribution of point cloud features and image features, resulting in mediocre performance. To address these issues, we propose a multisensor fusion network for unstructured scene segmentation with surface normal (SN) incorporated, called MF-SN Net. For the first factor, we effectively use complementary features to highlight the characteristics of unstructured scenes. Point represented and range view (RV) represented LiDAR information are combined as a baseline to fully utilize 3-D information while ensuring efficiency. RGB images from the camera are reweighted and fused with RV-represented LiDAR features at different scales. Second, we propose a novel method to reweight image features before fusing them into LiDAR features using SN information, which can effectively reduce the negative impact of inaccurate image features. Third, we design a cross-layer attention module to enhance semantic information from high-level features to different layers, which can optimize feature extraction from original point clouds. What is more, we make a synthesized dataset using CARLA simulator to enrich the experimental scenes, so that the network's performance can be evaluated in various conditions. Experimental results on different datasets demonstrate the effectiveness and robustness of our network, showing its competitiveness with state-of-the-art methods.

Moving Object Segmentation Network for Multi-View Fusion of Vehicle-Mounted LiDAR

ObjectFusion: an Object Detection and Segmentation Framework with RGB-D SLAM and Convolutional Neural Networks

Efficient Spatial-Temporal Information Fusion for LiDAR-Based 3D Moving Object Segmentation

Accurate 3D Semantic Segmentation of Point Clouds for Intelligent Vehicles Based on Multi-view Edge Guidance and Fusion

Accurate 3-D Semantic Segmentation of Point Clouds for Intelligent Vehicles Based on Multiview Edge Guidance and Fusion

Sequential Multi-view Fusion Network for Fast LiDAR Point Motion Estimation.

Enhancing Moving Object Segmentation with Spatio-Temporal Information Fusion

A Multi-view 3D Vehicle Detection Method Based On Novel 3D Proposal Generation Method

MotionBEV: Attention-Aware Online LiDAR Moving Object Segmentation with Bird's Eye View based Appearance and Motion Features

RGB and LiDAR Fusion-based 3D Semantic Segmentation for Autonomous Driving

Robust 3D Semantic Segmentation Based on Multi-Phase Multi-Modal Fusion for Intelligent Vehicles

Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions

Improved 3D Semantic Segmentation Model Based on RGB Image and LiDAR Point Cloud Fusion for Automantic Driving

MV-MOS: Multi-View Feature Fusion for 3D Moving Object Segmentation

Revisiting Multi-modal 3D Semantic Segmentation in Real-world Autonomous Driving

Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data

Dual-Branch Dynamic Object Segmentation Network Based on Spatio-Temporal Information Fusion

Multi-sensor Fusion Network for Unstructured Scene Segmentation with Surface Normal Incorporated

Multi-View Fusion of Sensor Data for Improved Perception and Prediction in Autonomous Driving

MENet: Multi-Modal Mapping Enhancement Network for 3D Object Detection in Autonomous Driving

Vehicle Motion Segmentation Via Combining Neural Networks and Geometric Methods