Abstract:Efficient point cloud visualization is indispensable for practical applications. In the context of point cloud visualization, 3-D rendering can be viewed as the kernel that transforms 3-D points into a 2-D scene image. Compared with traditional point-based rendering (PBR), neural image-based rendering (NIBR) has gradually emerged as a feasible solution for point cloud rendering. To efficiently render sparse and colorless mobile laser scanning (MLS) point cloud, we propose a novel neural rendering framework based on deep generative learning, named MLS point cloud rendering with generative adversarial network (MPR-GAN). In this framework, perspective projection with intrinsic parameter scaling and cumulative distribution normalization is first utilized to transform the 3-D point cloud into a compact 2-D image; a conditional generative adversarial network (CGAN)-based rendering model is then proposed to generate a photorealistic scene image from the projected 2-D image. In this CGAN model, the asymmetric encoder–decoder generator can implement inpainting and true colorization using context feature capturing and edge information perception; a multiscale discriminator is built to guarantee the model output with global consistency and local details. Moreover, a hybrid loss function is designed to improve the visual quality of the generated images with similarity constraints from both the content and the structure. Two public MLS point cloud datasets are selected and employed to carry out extensive evaluation using MPR-GAN and other baseline frameworks. The experimental results demonstrate that MPR-GAN achieves the state-of-the-art rendering performance in terms of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). Furthermore, the efficiency analysis shows that MPR-GAN can support real-time rendering, achieving end-to-end rendering from raw points.

PBR-GAN: Imitating Physically-Based Rendering with Generative Adversarial Networks

3D-Aware Image Synthesis Via Learning Structural and Textural Representations

PBR-Net: Imitating Physically Based Rendering using Deep Neural Network

Research on realistic effect generation algorithm of rendering images based on GAN

E-DBPN: Enhanced Deep Back-Projection Networks for Remote Sensing Scene Image Superresolution

MaterialGAN

MaterialGAN: Reflectance Capture using a Generative SVBRDF Model

Physically‐Based Lighting for 3D Generative Models of Cars

MPR-GAN: A Novel Neural Rendering Framework for MLS Point Cloud With Deep Generative Learning

PSGAN: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening

UPHDR-GAN: Generative Adversarial Network for High Dynamic Range Imaging with Unpaired Data

Physically Guided Generative Adversarial Network for Holographic 3D Content Generation from Multi-view Light Field

Staged Generative Adversarial Networks With Adversarial-Boundary

GP-GAN: Towards Realistic High-Resolution Image Blending

PUGAN: Physical Model-Guided Underwater Image Enhancement Using GAN with Dual-Discriminators

Progressively Unfreezing Perceptual GAN

Boosting 3D Object Generation through PBR Materials

GANeRF: Leveraging Discriminators to Optimize Neural Radiance Fields

GL-GAN: Adaptive Global and Local Bilevel Optimization model of Image Generation

HRPGAN: A GAN-based Model to Generate High-resolution Remote Sensing Images