Abstract:Efficient point cloud visualization is indispensable for practical applications. In the context of point cloud visualization, 3-D rendering can be viewed as the kernel that transforms 3-D points into a 2-D scene image. Compared with traditional point-based rendering (PBR), neural image-based rendering (NIBR) has gradually emerged as a feasible solution for point cloud rendering. To efficiently render sparse and colorless mobile laser scanning (MLS) point cloud, we propose a novel neural rendering framework based on deep generative learning, named MLS point cloud rendering with generative adversarial network (MPR-GAN). In this framework, perspective projection with intrinsic parameter scaling and cumulative distribution normalization is first utilized to transform the 3-D point cloud into a compact 2-D image; a conditional generative adversarial network (CGAN)-based rendering model is then proposed to generate a photorealistic scene image from the projected 2-D image. In this CGAN model, the asymmetric encoder–decoder generator can implement inpainting and true colorization using context feature capturing and edge information perception; a multiscale discriminator is built to guarantee the model output with global consistency and local details. Moreover, a hybrid loss function is designed to improve the visual quality of the generated images with similarity constraints from both the content and the structure. Two public MLS point cloud datasets are selected and employed to carry out extensive evaluation using MPR-GAN and other baseline frameworks. The experimental results demonstrate that MPR-GAN achieves the state-of-the-art rendering performance in terms of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). Furthermore, the efficiency analysis shows that MPR-GAN can support real-time rendering, achieving end-to-end rendering from raw points.

LM-GAN: A Photorealistic All-Weather Parametric Sky Model

Creating synthetic night-time visible-light meteorological satellite images using the GAN method

Real-time Generation of Infrared Dynamic Sky Scene

Generation of Multi-Spectral Scene Images under Different Weather Conditions

GANSky -- fast curved sky weak lensing simulations using Generative Adversarial Networks

Multispectral Band-Aware Generation of Satellite Images across Domains Using Generative Adversarial Networks and Contrastive Learning

A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-resolution

Deep Lighting Environment Map Estimation from Spherical Panoramas

EverLight: Indoor-Outdoor Editable HDR Lighting Estimation

Photographic Visualization of Weather Forecasts with Generative Adversarial Networks

MPR-GAN: A Novel Neural Rendering Framework for MLS Point Cloud With Deep Generative Learning

A Generative Model for Generic Light Field Reconstruction

Physically‐Based Lighting for 3D Generative Models of Cars

GAN-Based LiDAR Translation between Sunny and Adverse Weather for Autonomous Driving and Driving Simulation

LiDAR-3DGS: LiDAR Reinforced 3D Gaussian Splatting for Multimodal Radiance Field Rendering

Generating Physically-Consistent Satellite Imagery for Climate Visualizations

Generative Modeling of High-resolution Global Precipitation Forecasts

A Generative Deep Learning Approach to Stochastic Downscaling of Precipitation Forecasts

LidarDM: Generative LiDAR Simulation in a Generated World

Thin Cloud Removal Generative Adversarial Network Based on Sparse Transformer in Remote Sensing Images

The Sky's the Limit: Re-lightable Outdoor Scenes via a Sky-pixel Constrained Illumination Prior and Outside-In Visibility