Abstract:This paper proposes a regularizer called Implicit Neural Representation Regularizer (INRR) to improve the generalization ability of the Implicit Neural Representation (INR). The INR is a fully connected network that can represent signals with details not restricted by grid resolution. However, its generalization ability could be improved, especially with non-uniformly sampled data. The proposed INRR is based on learned Dirichlet Energy (DE) that measures similarities between rows/columns of the matrix. The smoothness of the Laplacian matrix is further integrated by parameterizing DE with a tiny INR. INRR improves the generalization of INR in signal representation by perfectly integrating the signal's self-similarity with the smoothness of the Laplacian matrix. Through well-designed numerical experiments, the paper also reveals a series of properties derived from INRR, including momentum methods like convergence trajectory and multi-scale similarity. Moreover, the proposed method could improve the performance of other signal representation methods.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **Improve the generalization ability of Implicit Neural Representation (INR) in signal representation, especially in the case of non - uniformly sampled data.** Specifically, the paper points out that although INR (Implicit Neural Representation) can represent the details of signals well and is not limited by grid resolution, its generalization ability needs to be improved, especially when dealing with non - uniformly sampled data. The author describes this problem in detail through the following points: 1. **Limitations of INR**: - INR can fit the training data well, but behaves unreasonably when predicting parts outside the training data. - The smoothness of INR is controlled by a hyper - parameter, and this hyper - parameter changes with different missing rates, making it difficult to find the optimal value. 2. **Deficiencies of existing methods**: - Traditional regularization methods such as L1 - norm, L2 - norm and Dropout cannot fully capture the structural features of images. - Existing regularization methods mainly focus on pixel - level losses and ignore the large - scale structural features of images. 3. **Proposed new method**: - The paper proposes a new regularizer - Implicit Neural Representation Regularizer (INRR), which aims to improve the generalization ability of INR by learning Dirichlet Energy (DE) to measure the self - similarity of images and the smoothness of the Laplacian matrix. - INRR parameterizes the Laplacian matrix in DE and uses a small INR network to learn the non - local similarity of images, enabling the model to better adapt to different types of missing patterns. 4. **Experimental verification**: - Through a series of numerical experiments, the paper shows the superior performance of INRR under different missing patterns, including random missing, block missing and texture missing. - The experimental results show that INRR is not only superior to other methods in quantitative indicators, but also performs well in qualitative visual effects. In summary, the main goal of this paper is to solve the generalization problem of INR on non - uniformly sampled data by introducing INRR, thereby improving its overall performance in signal representation tasks.

Regularize implicit neural representation by itself

Disorder-Invariant Implicit Neural Representation

RHINO: Regularizing the Hash-based Implicit Neural Representation

FINER: Flexible spectral-bias tuning in Implicit NEural Representation by Variable-periodic Activation Functions

Where Do We Stand with Implicit Neural Representations? A Technical and Performance Survey

ReLUs Are Sufficient for Learning Implicit Neural Representations

Boosting of Implicit Neural Representation-based Image Denoiser

Attention Beats Linear for Fast Implicit Neural Representation Generation

Conv-INR: Convolutional Implicit Neural Representation for Multimodal Visual Signals

DINER: Disorder-Invariant Implicit Neural Representation

A Structured Dictionary Perspective on Implicit Neural Representations

HOIN: High-Order Implicit Neural Representations

Revisiting Implicit Neural Representations in Low-Level Vision

HyperINR: A Fast and Predictive Hypernetwork for Implicit Neural Representations via Knowledge Distillation

Improving Generalization and Convergence by Enhancing Implicit Regularization

Optimizing 3D Geometry Reconstruction from Implicit Neural Representations

Inductive Gradient Adjustment For Spectral Bias In Implicit Neural Representations

Seeing Implicit Neural Representations as Fourier Series

FreqINR: Frequency Consistency for Implicit Neural Representation with Adaptive DCT Frequency Loss

Towards a Sampling Theory for Implicit Neural Representations