Applying Collaborative Adversarial Learning to Blind Point Cloud Quality Measurement

Jilong Wang,Wei Gao,Ge Li
DOI: https://doi.org/10.1109/TIM.2023.3322475
IF: 5.6
2023-01-01
IEEE Transactions on Instrumentation and Measurement
Abstract:As an emerging 3-D presentation format, point cloud plays a vital role in the field of immersive visual media, especially for applications related to human vision. The measurement of point cloud quality is of great significance for facilitating point cloud quality enhancement, optimizing the equipment chain of the point cloud, and improving human visual perception quality. In this article, we propose MMF-PCQM, a blind point cloud quality measurement network that leverages multimodal and multiscale features, i.e., multiscale texture features in 2-D images and geometric information in points, to assess the point cloud quality. MMF-PCQM consists of a preprocessing module, dual-branch fusion network module, and feature learning module. Specifically, the preprocessing module transfers a point cloud into 2-D images and 3-D point clusters. The dual-branch fusion network module aims to capture and integrate the multiscale complementary features from the multimodal data to attain the final quality measurement. The feature learning module bridges two modalities to enhance the fused feature through a collaborative adversarial learning strategy. We evaluate MMF-PCQM on two large publicly available datasets, including the Waterloo Point Cloud (WPC) dataset and the SIAT Point Cloud Quality Database (SIAT-PCQD). Extensive experimental results and ablation studies demonstrate that our MMF-PCQM achieves better results compared with the state-of-the-art methods.
What problem does this paper attempt to address?