Abstract:Urban management and survey departments have begun investigating the feasibility ofacquiring data from various laser scanning systems for urban infrastructure measurements and assessments. Roadside objects such as cars, trees, traffic poles, pedestrians, bicycles and e-bicycles describe the static and dynamic urban information available for acquisition. Because of the unstructured nature of 3D point clouds, the rich targets in complex road scenes, and the varying scales of roadside objects, finely classifying these roadside objects from various point clouds is a challenging task. In this paper, we integrate two representations of roadside objects, point clouds and multiview images to propose a point-group-view network named PGVNet for classifying roadside objects into cars, trees, traffic poles, and small objects (pedestrians, bicycles and e-bicycles) from generalized point clouds. To utilize the topological information of the point clouds, we propose a graph attention convolution operation called AtEdgeConv to mine the relationship among the local points and to extract local geometric features. In addition, we employ a hierarchical view-group-object architecture to diminish the redundant information between similar views and to obtain salient viewwise global features. To fuse the local geometric features from the point clouds and the global features from multiview images, we stack an attention-guided fusion network in PGVNet. In particular, we quantify and leverage the global features as an attention mask to capture the intrinsic correlation and discriminability of the local geometric features, which contributes to recognizing the different roadside objects with similar shapes. To verify the effectiveness and generalization of our methods, we conduct extensive experiments on six test datasets of different urban scenes, which were captured by different laser scanning systems, including mobile laser scanning (MLS) systems, unmanned aerial vehicle (UAV)-based laser scanning (ULS) systems and backpack laser scanning (BLS) systems. Experimental results, and comparisons with state-of-the-art methods, demonstrate that the PGVNet model is able to effectively identify various cars, trees, traffic poles and small objects from generalized point clouds, and achieves promising performances on roadside object classifications, with an overall accuracy of 95.76%. Our code is released on https://github.com/flidarcode/PGVNet.

A Convolutional Neural Network Method to Improve Efficiency and Visualization in Modeling Driver’s Visual Field on Roads Using MLS Data

Visual Localizer: Outdoor Localization Based on ConvNet Descriptor and Global Optimization for Visually Impaired Pedestrians

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

Multi-Dimensional Traffic Congestion Detection Based on Fusion of Visual Features and Convolutional Neural Network

Monocular BEV Perception of Road Scenes Via Front-to-Top View Projection

Research on Visual Perception Technology of Autonomous Driving Based on Improved Convolutional Neural Network

VPC-Net: Completion of 3D Vehicles from MLS Point Clouds

BCSSN: Bi-direction Compact Spatial Separable Network for Collision Avoidance in Autonomous Driving

A Deep Learning Based Framework for Blind Road Occupancy Detection

The Design of Preventive Automated Driving Systems Based on Convolutional Neural Network

A New Multi-Branch Convolutional Neural Network and Feature Map Extraction Method for Traffic Congestion Detection

NeurAll: Towards a Unified Visual Perception Model for Automated Driving

Simultaneous Road Surface and Centerline Extraction From Large-Scale Remote Sensing Images Using CNN-Based Segmentation and Tracing

Real-time Self-Driving Car Navigation and Obstacle Avoidance Using Mobile 3D Laser Scanner and GNSS

Automatic Outer Contour Detection and Quantification of Vehicles Using Monocular Vision

Mapping road safety features from streetview imagery: A deep learning approach

A joint deep learning network of point clouds and multiple views for roadside object classification from lidar point clouds

NVAutoNet: Fast and Accurate 360$^{\circ}$ 3D Visual Perception For Self Driving

LGmap: Local-to-Global Mapping Network for Online Long-Range Vectorized HD Map Construction

3D car-detection based on a Mobile Deep Sensor Fusion Model and real-scene applications

Simultaneous Viewpoint- and Condition-invariant Loop Closure Detection based on LiDAR Descriptor for Outdoor Large-scale Environments