Abstract:Neural Radiance Fields (NeRF) use multi-view images for 3D scene representation, demonstrating remarkable performance. As one of the primary sources of multi-view images, multi-camera systems encounter challenges such as varying intrinsic parameters and frequent pose changes. Most previous NeRF-based methods assume a unique camera and rarely consider multi-camera scenarios. Besides, some NeRF methods that can optimize intrinsic and extrinsic parameters still remain susceptible to suboptimal solutions when these parameters are poor initialized. In this paper, we propose MC-NeRF, a method that enables joint optimization of both intrinsic and extrinsic parameters alongside NeRF. The method also supports each image corresponding to independent camera parameters. First, we tackle coupling issue and the degenerate case that arise from the joint optimization between intrinsic and extrinsic parameters. Second, based on the proposed solutions, we introduce an efficient calibration image acquisition scheme for multi-camera systems, including the design of calibration object. Finally, we present an end-to-end network with training sequence that enables the estimation of intrinsic and extrinsic parameters, along with the rendering network. Furthermore, recognizing that most existing datasets are designed for a unique camera, we construct a real multi-camera image acquisition system and create a corresponding new dataset, which includes both simulated data and real-world captured images. Experiments confirm the effectiveness of our method when each image corresponds to different camera parameters. Specifically, we use multi-cameras, each with different intrinsic and extrinsic parameters in real-world system, to achieve 3D scene representation without providing initial poses.

CT-NeRF: Incremental Optimizing Neural Radiance Field and Poses with Complex Trajectory

IL-NeRF: Incremental Learning for Neural Radiance Fields with Camera Pose Alignment

CTNeRF: Cross-Time Transformer for Dynamic Neural Radiance Field from Monocular Video

CBARF: Cascaded Bundle-Adjusting Neural Radiance Fields from Imperfect Camera Poses

Pre-NeRF 360: Enriching Unbounded Appearances for Neural Radiance Fields

TD-NeRF: Novel Truncated Depth Prior for Joint Camera Pose and Neural Radiance Field Optimization

Blending Distributed NeRFs with Tri-stage Robust Pose Optimization

Pose-Free Neural Radiance Fields via Implicit Pose Regularization

Structure-aware neural radiance fields without posed camera

SiNeRF: Sinusoidal Neural Radiance Fields for Joint Pose Estimation and Scene Reconstruction

SN 2 eRF: A Framework for Neural Radiance Fields given Sparse and Noisy Poses

MC-NeRF: Multi-Camera Neural Radiance Fields for Multi-Camera Image Acquisition Systems

Enhance-NeRF: Multiple Performance Evaluation for Neural Radiance Fields

SG-NeRF: Neural Surface Reconstruction with Scene Graph Optimization

Structure-Aware NeRF without Posed Camera via Epipolar Constraint

NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction

UP-NeRF: Unconstrained Pose-Prior-Free Neural Radiance Fields

Robust SG-NeRF: Robust Scene Graph Aided Neural Surface Reconstruction

PI-NeRF: A Partial-Invertible Neural Radiance Fields for Pose Estimation

FDC-NeRF: Learning Pose-Free Neural Radiance Fields with Flow-Depth Consistency.

AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimization