Intelligent Point Cloud Processing, Sensing, and Understanding

Miaohui Wang,Guanghui Yue,Jian Xiong,Sukun Tian

DOI: https://doi.org/10.3390/s24010283

IF: 3.9

2024-01-03

Sensors

Abstract:Point clouds are considered one of the fundamental pillars for representing the 3D digital landscape [...]

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

What problem does this paper attempt to address?

Perspectives on Point Cloud-Based 3D Scene Modeling and XR Presentation Within the Cloud-Edge-client Architecture

Hongjia Wu,Hongxin Zhang,Jiang Cheng,Jianwei Guo,Wei Chen

DOI: https://doi.org/10.1016/j.visinf.2023.06.007

2023-01-01

Visual Informatics

Abstract:With the support of edge computing, the synergy and collaboration among central cloud, edge cloud, and terminal devices form an integrated computing ecosystem known as the cloud-edge-client architecture. This integration unlocks the value of data and computational power, presenting significant opportunities for large-scale 3D scene modeling and XR presentation. In this paper, we explore the perspectives and highlight new challenges in 3D scene modeling and XR presentation based on point cloud within the cloud-edge-client integrated architecture. We also propose a novel cloud-edge-client integrated technology framework and a demonstration of municipal governance application to address these challenges.
A New Approach of Point Cloud Processing and Scene Segmentation for Guiding the Visually Impaired

Kailun Yang,Kaiwei Wang,Ruiqi Cheng,Xunmin Zhu

DOI: https://doi.org/10.1049/cp.2015.0778

2015-01-01

Abstract:Point clouds of 3D scenes are widely applied in guiding the visually impaired by precious research. Many auxiliary systems for the visually impaired are integrated with RGB-D sensors such as Kinect and binocular cameras, which are able to acquire depth pictures and 3D point clouds. Real-time location of objects is adjusted to the world coordinate system through utilization of attitude angle transducers. This paper proposed a novel approach of scene segmentation based on the estimation of normal vectors of a point cloud. Multiplying a point cloud's normal vectors in two directions helps to eliminate correlation in different directions. It is used to split a stereo scene into several surfaces such as ground, walls and slopes. The method is faster and can obtain more separated results than RANSAC algorithm. Besides, three ways to evaluate surface smoothness are compared, including inconsistent degree of normal vectors, variance of depths and difference between normal vectors of two sizes of adjacent regions. Experimental results attained from indoor and outdoor circumstances are presented to validate the approach. It is demonstrated that the proposed method can be efficiently applied into scene segmentation and guiding the visually impaired.
Pass3d: Precise And Accelerated Semantic Segmentation For 3d Point Cloud

Xin Kong,Guangyao Zhai,Baoquan Zhong,Yong Liu

DOI: https://doi.org/10.1109/IROS40897.2019.8968296

2019-01-01

Abstract:In this paper, we propose PASS3D to achieve point-wise semantic segmentation for 3D point cloud. Our framework combines the efficiency of traditional geometric methods with robustness of deep learning methods, consisting of two stages: At stage-1, our accelerated cluster proposal algorithm will generate refined cluster proposals by segmenting point clouds without ground, capable of generating less redundant proposals with higher recall in an extremely short time; stage-2 we will amplify and further process these proposals by a neural network to estimate semantic label for each point and meanwhile propose a novel data augmentation method to enhance the network's recognition capability for all categories especially for non-rigid objects. Evaluated on KITTI raw dataset, PASS3D stands out against the state-of-the-art on some results, making itself competent to 3D perception in autonomous driving system. Our source code will be open-sourced. A video demonstration is available at https://www.youtube.com/watch?v=cukEqDuP_Qw
Massive Point Cloud Processing for Efficient Construction Quality Inspection and Control

Zhansheng Liu,Zehong Liu,Zhe Sun

DOI: https://doi.org/10.3390/s24216806

IF: 3.9

2024-10-24

Sensors

Abstract:The construction of large-scale civil infrastructures requires massive spatiotemporal data to support the management and control of scheduling, quality control, and safety monitoring. Existing artificial-intelligence-based data processing algorithms rely heavily on experienced engineers to adjust the parameters of data processing, which is inefficient and time-consuming when dealing with huge datasets. Limited studies have compared the performance of different algorithms on a unified dataset. This study proposes a framework and evaluation system for comparing different data processing policies for processing huge spatiotemporal data in construction quality control. The proposed method compares the combination of multiple types of algorithms involved in the processing of massive point cloud data. The performance of data processing strategies is evaluated through this framework, and the optimal point cloud processing strategies are explored based on registration accuracy and data fidelity. Results show that a reasonable choice of combinations of point cloud sampling, filtering, and registration algorithms can significantly improve the efficiency of point cloud data processing and satisfy engineering demands for data accuracy and completeness. The proposed method can be applied to the civil engineering problem of processing a large amount of point cloud data and selecting the optimal processing method.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Scene Segmentation and Understanding for Context-Free Point Clouds

A. Chalmers,Keith Bugeja,Kurt Debattista,Sandro Spina

DOI: https://doi.org/10.2312/PGS.20141244

Abstract:The continuous development of new commodity hardware intended to capture the surface structure of objects is quickly making point cloud data ubiquitous. Scene understanding methods address the problem of determining the objects present in a point cloud which, dependant on sensor capabilities and object occlusions, is normally noisy and incomplete. In this paper, we propose a novel technique which enables automatic identiﬁcation of semantically meaningful structures within point clouds acquired using different sensors on a variety of scenes. The representation model, namely the structure graph, with nodes representing planar surface segments, is computed over these point clouds to help with the identiﬁcation task. In order to accommodate for more complex objects (e.g. chair, couch, cabinet, table), a training process is used to determine and concisely describe, within each object’s structure graph, its important shape characteristics. Results on a variety of point clouds show how our method can quickly discern certain object types.

Computer Science,Engineering,Environmental Science
Study on the Application Status of 3D Point Cloud

Huimin Yuan,Ziqi Luo,Chuan Ding,Jiajun Ma

DOI: https://doi.org/10.9734/jerr/2024/v26i81250

2024-08-12

Journal of Engineering Research and Reports

Abstract:This paper aims to discuss the application and development trend of 3D point cloud technology in different fields. As an effective means of digitizing the real world, 3D point cloud has been widely used in many fields, including computer-aided design, building information modeling, geographic information systems, robot navigation, autonomous vehicles, and the digitalization of cultural relics. In this paper, the techniques of acquiring 3D point cloud data are summarized, including laser scanning, structured light scanning and time flight measurement. Then, the latest developments in point cloud data preprocessing were analyzed. Further, the challenges of 3D point cloud technology are discussed, such as large data volume, slow processing speed and high precision requirements. Finally, this paper looks forward to the future development direction of 3D point cloud technology, especially in the integration of artificial intelligence and machine learning technology application potential. Through this study, we expect to provide researchers and practitioners in related fields with a comprehensive understanding of 3D point cloud technology and promote its application in a wider range of fields.
A Three-Step Approach for TLS Point Cloud Classification.

Zhuqiang Li,Liqiang Zhang,Xiaohua Tong,Bo Du,Yuebin Wang,Liang Zhang,Zhenxin Zhang,Hao Liu,Jie Mei,Xiaoyue Xing,P. Takis Mathiopoulos

DOI: https://doi.org/10.1109/TGRS.2016.2564501

IF: 8.2

2016-01-01

IEEE Transactions on Geoscience and Remote Sensing

Abstract:The ability to classify urban objects in large urban scenes from point clouds efficiently and accurately still remains a challenging task today. A new methodology for the effective and accurate classification of terrestrial laser scanning (TLS) point clouds is presented in this paper. First, in order to efficiently obtain the complementary characteristics of each 3-D point, a set of point-based de...
Recent Advances and Perspectives in Deep Learning Techniques for 3D Point Cloud Data Processing

Zifeng Ding,Yuxuan Sun,Sijin Xu,Yan Pan,Yanhong Peng,Zebing Mao

DOI: https://doi.org/10.3390/robotics12040100

IF: 3.7

2023-07-11

Robotics

Abstract:In recent years, deep learning techniques for processing 3D point cloud data have seen significant advancements, given their unique ability to extract relevant features and handle unstructured data. These techniques find wide-ranging applications in fields like robotics, autonomous vehicles, and various other computer-vision applications. This paper reviews the recent literature on key tasks, including 3D object classification, tracking, pose estimation, segmentation, and point cloud completion. The review discusses the historical development of these methods, explores different model architectures, learning algorithms, and training datasets, and provides a comprehensive summary of the state-of-the-art in this domain. The paper presents a critical evaluation of the current limitations and challenges in the field, and identifies potential areas for future research. Furthermore, the emergence of transformative methodologies like PoinTr and SnowflakeNet is examined, highlighting their contributions and potential impact on the field. The potential cross-disciplinary applications of these techniques are also discussed, underscoring the broad scope and impact of these developments. This review fills a knowledge gap by offering a focused and comprehensive synthesis of recent research on deep learning techniques for 3D point cloud data processing, thereby serving as a useful resource for both novice and experienced researchers in the field.

English Else
Advanced Methods for Point Cloud Processing and Simplification

Pavel Chmelar,Lubos Rejfek,Tan N. Nguyen,Duy-Hung Ha

DOI: https://doi.org/10.3390/app10103340

2020-05-12

Applied Sciences

Abstract:Nowadays, mobile robot exploration needs a rangefinder to obtain a large number of measurement points to achieve a detailed and precise description of a surrounding area and objects, which is called the point cloud. However, a single point cloud scan does not cover the whole area, so multiple point cloud scans must be acquired and compared together to find the right matching between them in a process called registration method. This method requires further processing and places high demands on memory consumption, especially for small embedded devices in mobile robots. This paper describes a novel method to reduce the burden of processing for multiple point cloud scans. We introduce our approach to preprocess an input point cloud in order to detect planar surfaces, simplify space description, fill gaps in point clouds, and get important space features. All of these processes are achieved by applying advanced image processing methods in combination with the quantization of physical space points. The results show the reliability of our approach to detect close parallel walls with suitable parameter settings. More importantly, planar surface detection shows a 99% decrease in necessary descriptive points almost in all cases. This proposed approach is verified on the real indoor point clouds.
PointGS: Bridging and Fusing Geometric and Semantic Space for 3D Point Cloud Analysis

Chenru Jiang,Kaizhu Huang,Junwei Wu,Xinheng Wang,Jimin Xiao,Amir Hussain

DOI: https://doi.org/10.1016/j.inffus.2022.10.016

2022-01-01

SSRN Electronic Journal

Abstract:Directly processing 3D point cloud data becomes dominant in classification and segmentation tasks. Present mainstream point based methods usually focus on learning in either geometric space (i.e. PointNet++) or semantic space (i.e. DGCNN). Owing to the irregular and unordered data property of point cloud, these methods still suffer from drawbacks of either ambiguous local features aggregation in geometric space or poor global features extraction in semantic space. While few prior works address these two defects simultaneously by fusing information from the dual spaces, we make a first attempt to develop a synergistic framework, called PointGS. Leveraging both the strength of geometric structure and semantic representation, PointGS establishes a mutual supervision mechanism that can bridge the two spaces and fuse complementary information for better analyzing 3D point cloud data. Compared with existing popular networks, our work attains obvious performance improvement on all three mainstream tasks without any sophisticated operations.
Intelligent City 3D Modeling Model Based on Multisource Data Point Cloud Algorithm

Youping Wu,Zhihui Zhou

DOI: https://doi.org/10.1155/2022/6135829

2022-07-23

Journal of Function Spaces

Abstract:With the rapid development of smart cities, intelligent navigation, and autonomous driving, how to quickly obtain 3D spatial information of urban buildings and build a high-precision 3D fine model has become a key problem to be solved. As the two-dimensional mapping results have constrained various needs in people's social life, coupled with the concept of digital city and advocacy, making three-dimensional, virtualization and actualization become the common pursuit of people's goals. However, the original point cloud obtained is always incomplete due to reasons such as occlusion during acquisition and data density decreasing with distance, resulting in extracted boundaries that are often incomplete as well. In this paper, based on the study of current mainstream 3D model data organization methods, geographic grids and map service specifications, and other related technologies, an intelligent urban 3D modeling model based on multisource data point cloud algorithm is designed for the two problems of unified organization and expression of urban multisource 3D model data. A point cloud preprocessing process is also designed: point cloud noise reduction and downsampling to ensure the original point cloud geometry structure remain unchanged, while improving the point cloud quality and reducing the number of point clouds. By outputting to a common 3D format, the 3D model constructed in this paper can be applied to many fields such as urban planning and design, architectural landscape design, urban management, emergency disaster relief, environmental protection, and virtual tourism.

mathematics, applied
Scene Cognition Pattern of Point Cloud-Generalization Point Cloud

LIU Chun,JIA Shoujun,WU Hangbin,HUANG Wei,ZHENG Ning,AKRAM Akbar

DOI: https://doi.org/10.11947/j.agcs.2022.20220019

2022-01-01

Abstract:With the rapid development of sensor technology and observation platform, point cloud data that is viewed as primary data of remote sensing, has gradually become an important information carrier. Moreover, it plays an increasingly significant role in the national major strategic needs such as geological disaster situation awareness, natural resources quantitative investigation and road traffic safety services. At the same time, driven by point cloud observation equipment and national major strategic needs, spatial scenes have changed from perception to cognition, and new requirements for cognitive processing algorithms and computing power have also been put forward. Therefore, based on the basic framework of point cloud scene cognition, this paper analyzes the research status of multi-source point cloud coupled observation, summarizes the key progress of point cloud scene cognition and typical applications in major national strategic needs, and summarizes the main problems facing point cloud scene cognition at present. On this basis, this paper focuses on the cutting-edge challenges of cloud scene cognition, avoids the traditional Euclidean space and turns to the high-dimensional tensor manifold space for point cloud data processing, proposes the scientific concept and technical framework of generalized point cloud, and provides a new research idea for the algorithm and computing power of cognitive processing of point cloud scene.
Zoom to Perceive Better: No-reference Point Cloud Quality Assessment via Exploring Effective Multiscale Feature

Jilong Wang,Wei Gao,Ge Li

DOI: https://doi.org/10.1109/tcsvt.2024.3362369

IF: 5.859

2024-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Point clouds offer a novel 3D data representation that has proven pivotal in immersive visual media applications involving human perception. Developing objective point cloud quality assessment (PCQA) methods is imperative, as they can substantially reduce human evaluation costs and drive advancements for visual perceptual experiences in point cloud related applications. Point cloud quality assessment without reference remains challenging. Previous PCQA methods predominantly employ a fixed perceptual distance and often overlook the variability in quality perceived from different viewpoints, which impedes their effectiveness in multiscale or multi-granularity feature extraction and learning, particularly for deep neural networks. The single fixed observation distance fails to capture the multi-resolution characteristics intrinsic to human perception. Addressing this gap, in this paper, we introduce a novel no-reference PCQA method (MOD-PCQA) that integrates multiscale features to enhance point cloud quality perception across diverse scales and granularities. MOD-PCQA pioneers a viewpoint aware feature learning framework, capable of capturing visual features across various granularity levels, from fine to coarse. Specifically, we process and project point clouds into images from different viewpoints. Then, we extract multi-scale features under corresponding perspectives through three branch networks. Finally, we design an alternate learning strategy to optimize the feature extraction network to continuously refine the learned feature information from both inter-scale and intra-scale perspectives. Comprehensive experiments conducted on the SJTU-PCQA and WPC databases validate the superiority of our proposed model over state-of-the-art PCQA methods. Our method achieves optimal performance on both benchmarks by a significant margin, which comprehensively validates its effectiveness for the challenging PCQA task. The source code will be available at https://openi.pcl.ac.cn/OpenPointCloud/MOD-PCQA [1].

engineering, electrical & electronic
OpenPointCloud-V2

Yongchi Zhang,Wei Gao,Ge Li

DOI: https://doi.org/10.1145/3552457.3555729

2022-01-01

Abstract:Point cloud processing is a very significant research field, as 3D point cloud plays a vital part in visual applications. Considering that there is no work to uniformly arrange the point cloud processing, we build a deep learning based open-source algorithm library of point cloud processing (OpenPointCloud). This paper gives a fundamental overview and analysis of point cloud processing methods, mainly involving five point cloud sampling models, one for point cloud completion, and one for point cloud post-processing. We comprehensively introduce these methods in our algorithm library, and then provide additional implementation of machine learning libraries, such as the mainstream tensorflow, tensorlayer and pytorch. Comprehensive evaluation experiments on some point datasets to systematically compare the performance of all these point cloud processing algorithms in the library.
Parallel Point Clouds: Hybrid Point Cloud Generation and 3D Model Enhancement via Virtual–Real Integration

Yonglin Tian,Xiao Wang,Yu Shen,Zhongzheng Guo,Zilei Wang,Fei-Yue Wang

DOI: https://doi.org/10.3390/rs13152868

IF: 5

2021-07-22

Remote Sensing

Abstract:Three-dimensional information perception from point clouds is of vital importance for improving the ability of machines to understand the world, especially for autonomous driving and unmanned aerial vehicles. Data annotation for point clouds is one of the most challenging and costly tasks. In this paper, we propose a closed-loop and virtual–real interactive point cloud generation and model-upgrading framework called Parallel Point Clouds (PPCs). To our best knowledge, this is the first time that the training model has been changed from an open-loop to a closed-loop mechanism. The feedback from the evaluation results is used to update the training dataset, benefiting from the flexibility of artificial scenes. Under the framework, a point-based LiDAR simulation model is proposed, which greatly simplifies the scanning operation. Besides, a group-based placing method is put forward to integrate hybrid point clouds, via locating candidate positions for virtual objects in real scenes. Taking advantage of the CAD models and mobile LiDAR devices, two hybrid point cloud datasets, i.e., ShapeKITTI and MobilePointClouds, are built for 3D detection tasks. With almost zero labor cost on data annotation for newly added objects, the models (PointPillars) trained with ShapeKITTI and MobilePointClouds achieved 78.6% and 60.0% of the average precision of the model trained with real data on 3D detection, respectively.

environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary
Deep Learning on Point Clouds and Its Application: A Survey

Weiping Liu,Jia Sun,Wanyi Li,Ting Hu,Peng Wang

DOI: https://doi.org/10.3390/s19194188

IF: 3.9

2019-09-26

Sensors

Abstract:Point cloud is a widely used 3D data form, which can be produced by depth sensors, such as Light Detection and Ranging (LIDAR) and RGB-D cameras. Being unordered and irregular, many researchers focused on the feature engineering of the point cloud. Being able to learn complex hierarchical structures, deep learning has achieved great success with images from cameras. Recently, many researchers have adapted it into the applications of the point cloud. In this paper, the recent existing point cloud feature learning methods are classified as point-based and tree-based. The former directly takes the raw point cloud as the input for deep learning. The latter first employs a k-dimensional tree (Kd-tree) structure to represent the point cloud with a regular representation and then feeds these representations into deep learning models. Their advantages and disadvantages are analyzed. The applications related to point cloud feature learning, including 3D object classification, semantic segmentation, and 3D object detection, are introduced, and the datasets and evaluation metrics are also collected. Finally, the future research trend is predicted.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Computational Methods of Acquisition and Processing of 3D Point Cloud Data for Construction Applications

Qian Wang,Yi Tan,Zhongya Mei

DOI: https://doi.org/10.1007/s11831-019-09320-4

IF: 9.7

2019-02-20

Archives of Computational Methods in Engineering

Abstract:Abstract3D point cloud data from sensing technologies such as 3D laser scanning and photogrammetry are able to capture the 3D surface geometries of target objects in an accurate and efficient manner. Due to these advantages, the construction industry has been capturing 3D point cloud data of construction sites, construction works, and construction equipment to enable better decision making in construction project management. The captured point cloud data are utilized to reconstruct 3D building models, check construction quality, monitor construction progress, improve construction safety etc. throughout the project lifecycle from design to construction and facilities management phase. This paper aims to review the state-of-the-art methods to acquire and process 3D point cloud data for construction applications. The different approaches to 3D point cloud data acquisition are reviewed and compared including 3D laser scanning, photogrammetry, videogrammetry, RGB-D camera, and stereo camera. Furthermore, the processing methods of 3D point cloud data are reviewed according to the four common processing procedures including (1) data cleansing, (2) data registration, (3) data segmentation, and (4) object recognition. For each processing procedure, the different processing methods and algorithms are compared and discussed in detail, which provides a useful guidance to both researchers and industry practitioners for adopting point cloud data in the construction industry.

computer science, interdisciplinary applications,engineering, multidisciplinary,mathematics
A Survey of Topologically Structural Representation and Computation of 3D Point Cloud Data

ZHANG Yi-kuan,ZHANG Xiao-peng,ZHA Hong-bin,ZHANG Jiang-she

2008-01-01

Journal of Image and Graphics

Abstract:Topological structure plays an important role in simplification,transformation,surface reconstruction,compression and transmission,matching and retrieval,shape analysis and understanding,and applications of point cloud data.It is one class of the key characters of 3D point cloud data.Relevant concepts,methods and techniques of topologically structural representation and computation are surveyed and summarized in this paper.Typical techniques and their latest developments are remarked.Practical applications of topologically structural representation and computation of point cloud data and our some latest results are presented.Our perspectives of these problems and techniques are shown.
3d Point Cloud Scene Data Acquisition And Its Key Technologies For Scene Understanding

Li Yong,Tong Guofeng,Yang Jingchao,Zhang Liqiang,Peng Hao,Gao Huashuai

DOI: https://doi.org/10.3788/LOP56.040002

2019-01-01

Laser & Optoelectronics Progress

Abstract:Scene understanding is an important research content in information science. Compared with the two-dimensional (2D) data, the three-dimensional (3D) data has many advantages. At present, there are many ways to acquire the point clouds, and meanwhile the point clouds with different acquisition methods have different characteristics. In addition, there lacks a complete and systematic research review on the key techniques for 3D scenes understanding. Thus, the different methods for point cloud acquisition arc summarized, and the different point cloud data and related databases arc compared and analyzed as well. Based on the current research progress of 3D scene understanding, the techniques for point cloud filtering, feature extraction, point cloud segmentation, and point cloud semantic segmentation in 3D scene understanding are compared and summarized. By the summary of the domestic and foreign literatures published in recent years, the problems occurred in the key technologies for 3D scene understanding are condensed, and the development trend of the 3D scene understanding problems is prospected. The 3D scene understanding based on point clouds is widely used in many fields due to its richness of data. However, as for the scene understanding effect of 3D point clouds, especially the scene understanding of laser point clouds with color information, there are still many contents needed to be investigated in depth.

Intelligent Point Cloud Processing, Sensing, and Understanding

Perspectives on Point Cloud-Based 3D Scene Modeling and XR Presentation Within the Cloud-Edge-client Architecture

A New Approach of Point Cloud Processing and Scene Segmentation for Guiding the Visually Impaired

Pass3d: Precise And Accelerated Semantic Segmentation For 3d Point Cloud

Massive Point Cloud Processing for Efficient Construction Quality Inspection and Control

Scene Segmentation and Understanding for Context-Free Point Clouds

Study on the Application Status of 3D Point Cloud

A Three-Step Approach for TLS Point Cloud Classification.

Recent Advances and Perspectives in Deep Learning Techniques for 3D Point Cloud Data Processing

Advanced Methods for Point Cloud Processing and Simplification

PointGS: Bridging and Fusing Geometric and Semantic Space for 3D Point Cloud Analysis

Intelligent City 3D Modeling Model Based on Multisource Data Point Cloud Algorithm

Scene Cognition Pattern of Point Cloud-Generalization Point Cloud

Zoom to Perceive Better: No-reference Point Cloud Quality Assessment via Exploring Effective Multiscale Feature

OpenPointCloud-V2

Parallel Point Clouds: Hybrid Point Cloud Generation and 3D Model Enhancement via Virtual–Real Integration

Deep Learning on Point Clouds and Its Application: A Survey

Computational Methods of Acquisition and Processing of 3D Point Cloud Data for Construction Applications

A Survey of Topologically Structural Representation and Computation of 3D Point Cloud Data

3d Point Cloud Scene Data Acquisition And Its Key Technologies For Scene Understanding