Abstract:Implicit Neural Representations (INRs) have emerged as a paradigm in knowledge representation, offering exceptional flexibility and performance across a diverse range of applications. INRs leverage multilayer perceptrons (MLPs) to model data as continuous implicit functions, providing critical advantages such as resolution independence, memory efficiency, and generalisation beyond discretised data structures. Their ability to solve complex inverse problems makes them particularly effective for tasks including audio reconstruction, image representation, 3D object reconstruction, and high-dimensional data synthesis. This survey provides a comprehensive review of state-of-the-art INR methods, introducing a clear taxonomy that categorises them into four key areas: activation functions, position encoding, combined strategies, and network structure optimisation. We rigorously analyse their critical properties, such as full differentiability, smoothness, compactness, and adaptability to varying resolutions while also examining their strengths and limitations in addressing locality biases and capturing fine details. Our experimental comparison offers new insights into the trade-offs between different approaches, showcasing the capabilities and challenges of the latest INR techniques across various tasks. In addition to identifying areas where current methods excel, we highlight key limitations and potential avenues for improvement, such as developing more expressive activation functions, enhancing positional encoding mechanisms, and improving scalability for complex, high-dimensional data. This survey serves as a roadmap for researchers, offering practical guidance for future exploration in the field of INRs. We aim to foster new methodologies by outlining promising research directions for INRs and applications.

Inl: Implicit Non-Local Network

Adaptive Deep Spiking Neural Network with Global-Local Learning Via Balanced Excitatory and Inhibitory Mechanism

Non-Local Neural Networks With Grouped Bilinear Attentional Transforms

IM-LIF: Improved Neuronal Dynamics with Attention Mechanism for Direct Training Deep Spiking Neural Network

Non-local Neural Networks

Implicit Decouple Network for Efficient Pose Estimation

Unifying Nonlocal Blocks for Neural Networks

ELA: Efficient Local Attention for Deep Convolutional Neural Networks

Attention Beats Linear for Fast Implicit Neural Representation Generation

iiANET: Inception Inspired Attention Hybrid Network for efficient Long-Range Dependency

Disentangled Non-Local Neural Networks

Convolution in Convolution for Network in Network

Information Bottleneck Approach to Spatial Attention Learning

No-reference Image Quality Assessment Via Non-local Dependency Modeling.

Where Do We Stand with Implicit Neural Representations? A Technical and Performance Survey

DIANet: Dense-and-Implicit Attention Network

On the Connection between Local Attention and Dynamic Depth-wise Convolution

Convolutional Interaction Network for Natural Language Inference

Neural Architecture Search for Lightweight Non-Local Networks

Invertible Attention

High-Level Parallelism and Nested Features for Dynamic Inference Cost and Top-Down Attention