Abstract:We consider the problem of novel view synthesis from unposed images in a single feed-forward. Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS, where we further extend it to offer a practical solution that relaxes common assumptions such as dense image views, accurate camera poses, and substantial image overlaps. We achieve this through identifying and addressing unique challenges arising from the use of pixel-aligned 3DGS: misaligned 3D Gaussians across different views induce noisy or sparse gradients that destabilize training and hinder convergence, especially when above assumptions are not met. To mitigate this, we employ pre-trained monocular depth estimation and visual correspondence models to achieve coarse alignments of 3D Gaussians. We then introduce lightweight, learnable modules to refine depth and pose estimates from the coarse alignments, improving the quality of 3D reconstruction and novel view synthesis. Furthermore, the refined estimates are leveraged to estimate geometry confidence scores, which assess the reliability of 3D Gaussian centers and condition the prediction of Gaussian parameters accordingly. Extensive evaluations on large-scale real-world datasets demonstrate that PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to perform fast and high - quality novel view synthesis from uncalibrated images. Specifically, the authors propose a new framework named PF3plat (Pose - Free Feed - Forward 3D Gaussian Splatting), aiming to achieve efficient novel view synthesis with a single - forward pass from images without pose calibration. Existing methods usually rely on strict assumptions such as dense image views, accurate camera poses, and a large amount of image overlap, which limit their wide use in practical applications. ### Main problems 1. **Dense image views**: Many existing methods require a large number of input images to ensure high - quality reconstruction and novel view synthesis. 2. **Accurate camera poses**: Most methods rely on known or highly accurate camera pose data. 3. **Image overlap**: Existing techniques usually require a significant overlap area between images to ensure reconstruction quality. These problems are difficult to meet in real - world scenarios because randomly captured images often have sparse and far - apart viewpoints and lack accurate camera pose information. Therefore, there is an urgent need for a method that can quickly and effectively generate new viewpoints even with significant viewpoint changes when there are fewer image inputs. ### Solutions PF3plat solves the above problems in the following ways: - **Relaxing common assumptions**: No longer relying on dense image views, accurate camera poses, and a large amount of image overlap. - **Coarse alignment and fine - tuning**: Using pre - trained monocular depth estimation and visual correspondence models to achieve rough alignment of 3D Gaussian distributions, and then introducing a lightweight learnable module to refine depth and pose estimates. - **Geometry - aware confidence scoring**: Evaluating the reliability of 3D Gaussian centers, thereby conditioning the prediction of Gaussian parameters (such as opacity, covariance, and color). Through these improvements, PF3plat not only improves the quality of 3D reconstruction and novel view synthesis but also greatly enhances the ability to handle sparse and uncalibrated images, making this method more practical and efficient in practical applications. ### Summary The main contribution of PF3plat is that it provides a fast and high - performance solution for general - purpose novel view synthesis from uncalibrated images, and can complete training and inference without additional data (such as a collection of RGB images). The performance of this method on large - scale real - world datasets is better than that of existing methods, making it a new frontier technology.

PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting

No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images

GGRt: Towards Pose-free Generalizable 3D Gaussian Splatting in Real-time

MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images

PreF3R: Pose-Free Feed-Forward 3D Gaussian Splatting from Variable-length Image Sequence

FreeSplat: Generalizable 3D Gaussian Splatting Towards Free-View Synthesis of Indoor Scenes

FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting

Visual SLAM with 3D Gaussian Primitives and Depth Priors Enabling Novel View Synthesis

HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction

Epipolar-Free 3D Gaussian Splatting for Generalizable Novel View Synthesis

Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairs

TranSplat: Generalizable 3D Gaussian Splatting from Sparse Multi-View Images with Transformers

Look Gauss, No Pose: Novel View Synthesis using Gaussian Splatting without Accurate Pose Initialization

Splatfacto-W: A Nerfstudio Implementation of Gaussian Splatting for Unconstrained Photo Collections

SparseGS: Real-Time 360° Sparse View Synthesis using Gaussian Splatting

Object Gaussian for Monocular 6D Pose Estimation from Sparse Views

SfM-Free 3D Gaussian Splatting via Hierarchical Training

HybridGS: Decoupling Transients and Statics with 2D and 3D Gaussian Splatting

SplatFormer: Point Transformer for Robust 3D Gaussian Splatting

MVPGS: Excavating Multi-view Priors for Gaussian Splatting from Sparse Input Views

FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training