Abstract:As a dynamic and essential component in the road environment of urban scenarios, vehicles are the most popular investigation targets. To monitor their behavior and extract their geometric characteristics, an accurate and instant measurement of vehicles plays a vital role in traffic and transportation fields. Point clouds acquired from the mobile laser scanning (MLS) system deliver 3D information of road scenes with unprecedented detail. They have proven to be an adequate data source in the fields of intelligent transportation and autonomous driving, especially for extracting vehicles. However, acquired 3D point clouds of vehicles from MLS systems are inevitably incomplete due to object occlusion or self-occlusion. To tackle this problem, we proposed a neural network to synthesize complete, dense, and uniform point clouds for vehicles from MLS data, named Vehicle Points Completion-Net (VPC-Net). In this network, we introduce a new encoder module to extract global features from the input instance, consisting of a spatial transformer network and point feature enhancement layer. Moreover, a new refiner module is also presented to preserve the vehicle details from inputs and refine the complete outputs with fine-grained information. Given sparse and partial point clouds as inputs, the network can generate complete and realistic vehicle structures and keep the fine-grained details from the partial inputs. We evaluated the proposed VPC-Net in different experiments using synthetic and real-scan datasets and applied the results to 3D vehicle monitoring tasks. Quantitative and qualitative experiments demonstrate the promising performance of the proposed VPC-Net and show state-of-the-art results.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the incompleteness of vehicle 3D point clouds in the point cloud data obtained by Mobile Laser Scanning (MLS) systems due to object occlusion or self - occlusion. Specifically, the paper proposes a neural network model named Vehicle Points Completion - Net (VPC - Net), aiming to generate complete, dense, and uniformly - distributed vehicle 3D point clouds from partial and sparse point cloud inputs. This helps to improve the application performance in fields such as 3D traffic monitoring and autonomous driving, because the complete geometry is crucial for 3D perception tasks (such as instance extraction, type classification, and trajectory estimation). ### Background and Problem Description of the Paper In the urban road environment, vehicles are one of the most popular research objects and also a dynamic and crucial component. In order to monitor their behaviors and extract their geometric features, accurate and immediate measurement of vehicles plays an important role in the traffic and transportation fields. The point clouds obtained by Mobile Laser Scanning (MLS) systems provide detailed 3D information of road scenes. This data source is particularly important in the fields of intelligent transportation and autonomous driving, especially in vehicle extraction. However, due to object occlusion or self - occlusion, the vehicle 3D point clouds obtained by MLS systems are inevitably incomplete. For example, in Figure 1a, some typical vehicle point clouds from the KITTI dataset are shown, and the missing parts in the vehicle scan point clouds can be clearly seen. This incompleteness of data seriously affects the potential uses of vehicle point clouds, because it actually changes the shape dimensions, shifts the object volume, and destroys the surface topology. ### Solution To solve this problem, the paper proposes a neural network model named VPC - Net, which can generate complete, realistic, and fine - grained - detail - preserving vehicle structures from partial and sparse point cloud inputs. The main contributions of VPC - Net include: 1. **End - to - End Network Design**: VPC - Net can directly process partial and sparse point clouds to generate uniform, dense, and complete point clouds. 2. **New Encoder Module**: The Spatial Transformer Network (STN) and Point Feature Enhancement Layer (PFE) are introduced to better extract global features. 3. **New Refinement Module**: By combining the partial input and the coarse point cloud generated by the decoder, the output is further optimized to preserve the original details of the input point cloud. 4. **Experimental Verification**: Experiments were carried out on the synthetic dataset (ShapeNet) and two real MLS datasets (KITTI and TUM - MLS - 2016), and the results show that VPC - Net outperforms the baseline methods. ### Model Architecture The architecture of VPC - Net includes three main modules: 1. **Encoder Module**: Used to extract global features from the original sparse point clouds. 2. **Decoder Module**: Converts the extracted global features into a coarse but complete point cloud, and then generates a dense point cloud. 3. **Refinement Module**: Combines the partial input and the dense point cloud generated by the decoder to further optimize the output and generate the final complete point cloud. ### Experimental Results The experimental results show that VPC - Net performs excellently in completing 3D vehicle point clouds. It can generate uniformly - distributed complete point clouds while maintaining the original details of the input point clouds. These results are verified not only on the synthetic dataset but also on the real MLS datasets, showing the potential of this method in practical applications. In conclusion, VPC - Net provides an effective method to solve the problem of incomplete vehicle 3D point clouds obtained by MLS systems, and provides important technical support for fields such as 3D traffic monitoring and autonomous driving.

VPC-Net: Completion of 3D Vehicles from MLS Point Clouds

COMPLETION OF SPARSE AND PARTIAL POINT CLOUDS OF VEHICLES USING A NOVEL END-TO-END NETWORK

SSF: Sparse Point Cloud Object Detection Based on Self-Adaptive Voxel Encoding and Focal-Sparse Convolution

Dual-scale Point Cloud Completion Network Based on High-Frequency Feature Fusion

Gap Completion in Point Cloud Scene occluded by Vehicles using SGC-Net

A semantic segmentation method for vehicle‐borne laser scanning point clouds in motorway scenes

A joint deep learning network of point clouds and multiple views for roadside object classification from lidar point clouds

Self-supervised Point Cloud Completion on Real Traffic Scenes via Scene-concerned Bottom-up Mechanism

SegVoxelNet: Exploring Semantic Context and Depth-aware Features for 3D Vehicle Detection from Point Cloud

VSL-Net: Voxel structure learning for 3D object detection

VP-Net: Voxels as Points for 3D Object Detection

VirPNet: A Multimodal Virtual Point Generation Network for 3D Object Detection

Capsule-Based Networks for Road Marking Extraction and Classification From Mobile LiDAR Point Clouds

MSPV3D: Multi-Scale Point-Voxels 3D Object Detection Net

CFPC: The Curbed Fake Point Collector to Pseudo-LiDAR-Based 3D Object Detection for Autonomous Vehicles

PV-RCNN++: Semantical Point-Voxel Feature Interaction for 3D Object Detection

Voxel-RCNN-Complex: An Effective 3-D Point Cloud Object Detector for Complex Traffic Conditions

Variational Relational Point Completion Network for Robust 3D Classification

3-D Vehicle Detection Enhancement Using Tracking Feedback in Sparse Point Clouds Environments

Channelwise and Spatially Guided Multimodal Feature Fusion Network for 3-D Object Detection in Autonomous Vehicles