Double-view Feature Fusion Network for LiDAR Semantic Segmentation
Sun Liujie,Zeng Tengfei,Fan Jingxing,Wang Wenju
DOI: https://doi.org/10.11834/jig.220943
2024-01-01
Journal of Image and Graphics
Abstract:Objective Point cloud semantic segmentation,as the basic technology of 3D point cloud data target detection,point cloud classification,and other projects,is an important part of the current 3D computer vision.At the same time,point cloud segmentation technology is the key for the computer to understand scenes,and it has been widely used in many fields such as autonomous driving,robotics,and augmented reality.Point cloud semantic segmentation refers to the point-by-point classification operation of points in the point cloud scene,that is,to judge the category of each point in the point cloud and finally segment and integrate accordingly.Generally,point cloud semantic segmentation technology can be divided into two categories according to different application scenarios:small-scale point cloud semantic segmentation and large-scale point cloud semantic segmentation.Small-scale point cloud semantic segmentation only performs semantic seg-mentation operations on indoor point cloud scenes or small-scale point cloud scenes,whereas the large-scale point cloud semantic segmentation replaces the deployment environment of the algorithm with outdoor large-scale point cloud data.Classification and integration for point clouds are usually performed on driving scenes or urban scenes.Compared with the point cloud semantic segmentation of small scenes,the semantic segmentation of large-scale point clouds has a wider range of applications and is extensively used in driving scene understanding,urban scene reconstruction,and other fields.How-ever,due to the large amount of data and the complexity of point cloud data,the task of semantic segmentation for point cloud in large scenes is more difficult.To improve the extraction quality of point features in a large-scale point cloud,a semantic segmentation method based on double-view feature fusion network for LiDAR semantic segmentation is proposed.Method Our method is composed of two parts,double-view feature fusion module and feature integration based on asym-metric convolution.In the down sampling stage,a double-view feature fusion module,which includes a double-view point cloud feature extraction module and a feature fusion block,is suggested.The double-view feature fusion module combines the cylindrical feature with the global feature of key points to reduce the feature loss caused by downsampling.The features in different views of the point cloud are combined by feature splicing in this module.Finally,the combined point cloud fea-tures are placed into the feature fusion block for feature dimensionality reduction and fusion.In the feature integration stage,a point cloud feature integration module is proposed based on asymmetric convolution,including asymmetric convo-lution and multiscale dimension-decomposition context modeling,achieving the enhancement and reconstruction of point cloud features by the operation of asymmetric point cloud feature processing and multi-scale context feature integration.The feature integration processes the double-view feature by asymmetric convolution and then uses multi-dimensional convo-lution and multiscale feature integration for feature optimization.Result In our experimental environment,our algorithm has the second-highest frequency weighted intersection over union accuracy rate and the highest mean intersection over union(mIoU)accuracy rate among recent algorithms.Our work focuses on the improvement of segmentation accuracy and achieves the highest segmentation accuracy in multiple categories.In vehicle categories such as cars and trucks,our method achieves a high segmentation accuracy.In categories such as bicycles,motorcycles,and pedestrians with small individuals and complex shapes,our method performs better than other methods.In buildings,railings,vegetation,and other categories that are at the edge of the point cloud scene and where the point cloud distribution is relatively sparse,the double-view feature fusion module in our method not only retains the geometric structure of the point cloud but also extracts the global features of the data,thereby achieving the high-precision segmentation of these categories.Our method achieves 63.9％mIoU on the SemanticKITTI dataset and leads the open-source segmentation methods.Compared with CA3DCNet,our method also achieves better segmentation results on the nuScenes dataset,and the mIoU accuracy is improved by 0.7％.Conclusion Our method achieves a high-precision semantic segmentation for a large-scale point cloud.A double-view feature fusion network is proposed for LiDAR semantic segmentation,which is suitable for the segmentation for a large-scale point cloud.Experimental results show the double-view feature fusion module can reduce the loss of edge information in a large-scale point cloud,thereby improving the segmentation accuracy of edge object in the scene.Experiments prove the feature integration module based on asymmetric convolution can effectively segment small-sized objects,such as pedes-trians,bicycles,motorcycles,and other categories.Our method is compared with a variety of semantic segmentation meth-ods for large-scale point cloud.In terms of accuracy,our method performs better and achieves an average segmentation accuracy of 63.9％on the SemanticKITTI dataset.