Semantic Point Cloud Upsampling

Zhuangzi Li,Ge Li,Thomas H. H. Li,Shan Liu,Wei Gao

DOI: https://doi.org/10.1109/tmm.2022.3160604

IF: 7.3

2023-01-01

IEEE Transactions on Multimedia

Abstract:Downsampled sparse point clouds are beneficial for data transmission and storage, but they are detrimental for semantic tasks due to information loss. In this paper, we examine an upsampling methodology that significantly reconstructs sparse clouds’ semantic representations. Specifically, we propose a novel semantic point cloud upsampling (SPU) framework for sparse point cloud classification. An SPU consists of two networks, i.e. an upsampling network and a classification network. They are skillfully unified to intensify semantic representations acting on the upsampling process. In the upsampling network, we first propose a novel graph aggregation convolution to construct hierarchical relations on sparse point clouds. To enhance stability and diversity during point upsampling, we then combine point shuffling and pre-interpolation technologies to build an enhanced upsampling module. Furthermore, we adopt the semantic prior information provided by a sparse point cloud to enhance its upsampling quality. The prior information is applied to an attention mechanism that can highlight key positions of the point cloud. We investigate different loss functions and conduct experiments on classical deep point networks, which effectively demonstrate the promising performance of our framework.