Abstract:Implicit Neural Representations (INRs) have emerged as a paradigm in knowledge representation, offering exceptional flexibility and performance across a diverse range of applications. INRs leverage multilayer perceptrons (MLPs) to model data as continuous implicit functions, providing critical advantages such as resolution independence, memory efficiency, and generalisation beyond discretised data structures. Their ability to solve complex inverse problems makes them particularly effective for tasks including audio reconstruction, image representation, 3D object reconstruction, and high-dimensional data synthesis. This survey provides a comprehensive review of state-of-the-art INR methods, introducing a clear taxonomy that categorises them into four key areas: activation functions, position encoding, combined strategies, and network structure optimisation. We rigorously analyse their critical properties, such as full differentiability, smoothness, compactness, and adaptability to varying resolutions while also examining their strengths and limitations in addressing locality biases and capturing fine details. Our experimental comparison offers new insights into the trade-offs between different approaches, showcasing the capabilities and challenges of the latest INR techniques across various tasks. In addition to identifying areas where current methods excel, we highlight key limitations and potential avenues for improvement, such as developing more expressive activation functions, enhancing positional encoding mechanisms, and improving scalability for complex, high-dimensional data. This survey serves as a roadmap for researchers, offering practical guidance for future exploration in the field of INRs. We aim to foster new methodologies by outlining promising research directions for INRs and applications.

On the Equivalence Between Implicit and Explicit Neural Networks: A High-dimensional Viewpoint

Deep Equilibrium Models are Almost Equivalent to Not-so-deep Explicit Models for High-dimensional Gaussian Mixtures

Explicitising The Implicit Intrepretability of Deep Neural Networks Via Duality

On the computation of the gradient in implicit neural networks

Quasi-Equivalence of Width and Depth of Neural Networks

Inl: Implicit Non-Local Network

Transformations establishing equivalence across neural networks: When have two networks learned the same task?

Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity

Subhomogeneous Deep Equilibrium Models

Exploring the Complexity of Deep Neural Networks through Functional Equivalence

Optimization Induced Equilibrium Networks: an Explicit Optimization Perspective for Understanding Equilibrium Models.

Deep neural networks with data dependent implicit activation function

Breaking the Curse of Dimensionality with Convex Neural Networks

A Generalization Bound for a Family of Implicit Networks

Optimizing Neural Networks in the Equivalent Class Space.

Low dimensional approximation and generalization of multivariate functions on smooth manifolds using deep ReLU neural networks

Two Sides of The Same Coin: Bridging Deep Equilibrium Models and Neural ODEs via Homotopy Continuation

A discretization-invariant extension and analysis of some deep operator networks

Where Do We Stand with Implicit Neural Representations? A Technical and Performance Survey

Convergence analysis of wide shallow neural operators within the framework of Neural Tangent Kernel

Wide Neural Networks as Gaussian Processes: Lessons from Deep Equilibrium Models