Abstract:In minimally-invasive brain surgeries with indirect and narrow operating environments, 3D brain reconstruction is crucial. However, as requirements of accuracy for some new minimally-invasive surgeries (such as brain-computer interface surgery) are higher and higher, the outputs of conventional 3D reconstruction, such as point cloud (PC), are facing the challenges that sample points are too sparse and the precision is insufficient. On the other hand, there is a scarcity of high-density point cloud datasets, which makes it challenging to train models for direct reconstruction of high-density brain point clouds. In this work, a novel model named stereoscopic-aware graph generative adversarial network (SG-GAN) with two stages is proposed to generate fine high-density PC conditioned on a single image. The Stage-I GAN sketches the primitive shape and basic structure of the organ based on the given image, yielding Stage-I point clouds. The Stage-II GAN takes the results from Stage-I and generates high-density point clouds with detailed features. The Stage-II GAN is capable of correcting defects and restoring the detailed features of the region of interest (ROI) through the up-sampling process. Furthermore, a parameter-free-attention-based free-transforming module is developed to learn the efficient features of input, while upholding a promising performance. Comparing with the existing methods, the SG-GAN model shows superior performance in terms of visual quality, objective measurements, and performance in classification, as demonstrated by comprehensive results measured by several evaluation metrics including PC-to-PC error and Chamfer distance.

Self-supervised single-view 3D point cloud reconstruction through GAN inversion

2D GANs Meet Unsupervised Single-view 3D Reconstruction

3D-Mask-GAN:Unsupervised Single-View 3D Object Reconstruction

Semi-supervised Three-dimensional Reconstruction Framework with GAN.

Monocular 3D Object Reconstruction with GAN Inversion

PointInverter: Point Cloud Reconstruction and Editing via a Generative Model with Shape Priors

High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization

Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversion

Weakly-Supervised Single-view Dense 3D Point Cloud Reconstruction Via Differentiable Renderer

Progressive Learning of 3D Reconstruction Network from 2D GAN Data

GAN Inversion Based Point Clouds Denoising in Foggy Scenarios for Autonomous Driving

Cross-domain Point Cloud Completion for Multi-Class Indoor Incomplete Objects Via Class-Conditional GAN Inversion

3D GAN Inversion with Pose Optimization

Semi-Supervised Generative Adversarial Model for 3D Recognition

Single-Image 3-D Reconstruction: Rethinking Point Cloud Deformation

SG-GAN: Fine Stereoscopic-Aware Generation for 3D Brain Point Cloud Up-sampling from a Single Image

PointCG: Self-supervised Point Cloud Learning via Joint Completion and Generation

Unsupervised 3D Point Cloud Reconstruction Via Exploring Multi-View Consistency and Complementarity

3D GAN Inversion with Facial Symmetry Prior

Weakly Supervised Part-wise 3D Shape Reconstruction from Single-View RGB Images

Single View Point Cloud Generation via Unified 3D Prototype