Noise-NeRF: Hide Information in Neural Radiance Fields using Trainable Noise

Qinglong Huang,Haoran Li,Yong Liao,Yanbin Hao,Pengyuan Zhou

2024-10-01

Abstract:Neural Radiance Field (NeRF) has been proposed as an innovative advancement in 3D reconstruction techniques. However, little research has been conducted on the issues of information confidentiality and security to NeRF, such as steganography. Existing NeRF steganography solutions have shortcomings in low steganography quality, model weight damage, and limited amount of steganographic information. This paper proposes Noise-NeRF, a novel NeRF steganography method employing Adaptive Pixel Selection strategy and Pixel Perturbation strategy to improve the quality and efficiency of steganography via trainable noise. Extensive experiments validate the state-of-the-art performances of Noise-NeRF on both steganography quality and rendering quality, as well as effectiveness in super-resolution image steganography.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper attempts to solve several key problems in NeRF (Neural Radiance Field) in terms of information hiding (i.e., steganography):

1. **Low - quality steganography**: Existing NeRF steganography methods lead to unstable rendering quality when embedding secret information, especially from different viewing angles. This is mainly because these methods embed information by retraining model parameters, which inevitably damages the model weights.

2. **Limited amount of information**: Current methods can only embed a small amount of information in a single image or binary code, and when too much information is embedded, the image quality will decline significantly.

3. **Ignoring high - resolution images**: Existing NeRF steganography methods are mainly applicable to low - quality images, and their steganography capabilities in high - resolution images have not been fully studied.

To solve these problems, the author proposes a new method named Noise - NeRF. This method uses trainable noise for information hiding instead of directly modifying the model weights. Specifically, Noise - NeRF introduces two strategies to improve steganography quality and efficiency:

- **Adaptive Pixel Selection strategy**: Select pixels according to the gradient difference to update the noise and ensure steganography accuracy.
- **Pixel Perturbation strategy**: Accelerate convergence, making the noise deviate from the original prediction faster, thereby improving steganography efficiency.

Through these innovations, Noise - NeRF can achieve high - quality and efficient information hiding without compromising the NeRF rendering quality, especially performing excellently in high - resolution images.

### Formula Summary

1. **Representation form of NeRF**:
\[
F_\theta: (x, y, z, \psi, \phi) \rightarrow (c, \sigma)
\]
where \((x, y, z)\) are spatial coordinates, \((\psi, \phi)\) are direction information, \(c\) is color information, and \(\sigma\) is volume density information.

2. **Volume rendering formula**:
\[
\hat{C}(r) = \sum_{i = 1}^{N} T_i(1 - \exp(-\sigma_i\delta_i))c_i
\]
where
\[
T_i=\exp\left(-\sum_{j = 1}^{i - 1}\sigma_j\delta_j\right)
\]

3. **Loss function**:
\[
L=\begin{cases}
\lambda_1\cdot L_{rgb}+\lambda_2\cdot L_{perturb}, & \text{if iteration}\leq\mu\\
L_{rgb}, & \text{if iteration}>\mu
\]
where
\[
L_{rgb}=\sum_{r\in R}\sum_{p\in r}\left\|\hat{C}_f(\gamma(p)+\delta)-C_M(\gamma(p))\right\|_2^2
\]
\[
L_{perturb}=-\sum_{r\in R}\sum_{p\in r}\left\|\hat{C}_f(\gamma(p)+\delta)-\hat{C}_f(\gamma(p))\right\|_2^2
\]

4. **Noise update formula**:
\[
\delta_i^p=\delta_{i - 1}^p+\eta\cdot(\nabla_{\delta_t^p}L_{rgb})
\]