Abstract:We propose 6DGS to estimate the camera pose of a target RGB image given a 3D Gaussian Splatting (3DGS) model representing the scene. 6DGS avoids the iterative process typical of analysis-by-synthesis methods (e.g. iNeRF) that also require an initialization of the camera pose in order to converge. Instead, our method estimates a 6DoF pose by inverting the 3DGS rendering process. Starting from the object surface, we define a radiant Ellicell that uniformly generates rays departing from each ellipsoid that parameterize the 3DGS model. Each Ellicell ray is associated with the rendering parameters of each ellipsoid, which in turn is used to obtain the best bindings between the target image pixels and the cast rays. These pixel-ray bindings are then ranked to select the best scoring bundle of rays, which their intersection provides the camera center and, in turn, the camera rotation. The proposed solution obviates the necessity of an "a priori" pose for initialization, and it solves 6DoF pose estimation in closed form, without the need for iterations. Moreover, compared to the existing Novel View Synthesis (NVS) baselines for pose estimation, 6DGS can improve the overall average rotational accuracy by 12% and translation accuracy by 22% on real scenes, despite not requiring any initialization pose. At the same time, our method operates near real-time, reaching 15fps on consumer hardware.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is **6 - Degree - of - Freedom (6DoF) camera pose estimation under a single - image and 3D Gaussian point - painting model**. Specifically, the paper proposes a method named 6DGS, aiming to estimate the camera pose of the target RGB image by given a 3D Gaussian point - painting (3DGS) model. The main innovation of the 6DGS method is that it avoids the iterative process required by traditional analysis - synthesis methods (such as iNeRF) and the requirement of the initial camera pose, but directly estimates the 6DoF pose by reversing the 3DGS rendering process. ### Main Problems and Solutions 1. **Problem**: Existing 6D camera pose estimation methods based on Neural Radiance Field (NeRF) (such as iNeRF) usually require a nearly correct initial camera pose to converge, and these methods rely on the iterative process, which leads to problems of low computational efficiency and being prone to getting stuck in local minima. 2. **Solutions**: - **6DGS Method**: By introducing a new ray - generation mechanism - Ellicell, rays are uniformly generated from the ellipsoidal surface of the 3D Gaussian point - painting model. Each ray is associated with the rendering parameters of the ellipsoid and is used to obtain the best binding between the target - image pixels and the projected rays. - **Attention Mechanism**: Use the learned attention map to select the best ray bundles. The intersections of these ray bundles provide the camera center position, and then the camera rotation is estimated. - **Closed - Form Solution**: 6DGS solves the ray intersections by the weighted least - squares method (wLS), thereby estimating the 6DoF camera pose in a closed - form without the need for an initial pose. ### Technical Details - **Ellicell Generation**: Divide the ellipsoidal surface into equal - area cells, and generate one ray from each cell. The direction of the ray is from the center of the ellipsoid to the center of the cell. - **Ray - Pixel Binding**: Use the DINOv2 pre - trained visual feature extractor to generate image features, and generate ray features through a multi - layer perceptron (MLP) and position encoding. The attention mechanism is used to match ray features and image features and select the ray bundles with the highest scores. - **Pose Estimation**: Solve the intersections of the selected rays by the weighted least - squares method to obtain the translation and rotation parameters of the camera. ### Experimental Results - **Dataset**: The paper conducted experiments on two real - world datasets, namely Tanks & Temples and Mip - NeRF 360°. - **Performance Comparison**: Compared with existing NeRF - based 6D pose estimation methods, 6DGS improves the average rotation accuracy by 12% and the translation accuracy by 22% without providing an initial pose. - **Real - Time Performance**: 6DGS can achieve a processing speed of 15 frames per second on consumer - grade hardware and is suitable for near - real - time applications. ### Summary 6DGS solves the problems of requiring an initial pose and an iterative process in traditional methods through innovative ray generation and attention mechanisms, achieving more efficient and accurate 6D camera pose estimation. This method has significant advantages in practical applications, especially in the absence of initial - pose priors.

6DGS: 6D Pose Estimation from a Single Image and a 3D Gaussian Splatting Model

6DOPE-GS: Online 6D Object Pose Estimation using Gaussian Splatting

Learning Stereopsis from Geometric Synthesis for 6D Object Pose Estimation

GS-Pose: Generalizable Segmentation-based 6D Object Pose Estimation with 3D Gaussian Splatting

GS2Pose: Two-stage 6D Object Pose Estimation Guided by Gaussian Splatting

KeyGS: A Keyframe-Centric Gaussian Splatting Method for Monocular Image Sequences

Camera Pose Estimation Using a 3D Gaussian Splatting Radiance Field

Look Gauss, No Pose: Novel View Synthesis using Gaussian Splatting without Accurate Pose Initialization

Object Gaussian for Monocular 6D Pose Estimation from Sparse Views

SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again

GSLoc: Efficient Camera Pose Refinement via 3D Gaussian Splatting

GGRt: Towards Pose-free Generalizable 3D Gaussian Splatting in Real-time

ZeroGS: Training 3D Gaussian Splatting from Unposed Images

Free-DyGS: Camera-Pose-Free Scene Reconstruction based on Gaussian Splatting for Dynamic Surgical Videos

No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images

Real-time 6D pose estimation from a single RGB image.

Estimating 6D Pose From Localizing Designated Surface Keypoints.

PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting

GGRt: Towards Generalizable 3D Gaussians Without Pose Priors in Real-Time

SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation

6D Object Pose Estimation from Approximate 3D Models for Orbital Robotics