Abstract:We introduce VPNet, a novel model-driven neural network architecture based on variable projection (VP). Applying VP operators to neural networks results in learnable features, interpretable parameters, and compact network structures. This paper discusses the motivation and mathematical background of VPNet and presents experiments. The VPNet approach was evaluated in the context of signal processing, where we classified a synthetic dataset and real electrocardiogram (ECG) signals. Compared to fully connected and one-dimensional convolutional networks, VPNet offers fast learning ability and good accuracy at a low computational cost of both training and inference. Based on these advantages and the promising results obtained, we anticipate a profound impact on the broader field of signal processing, in particular on classification, regression and clustering problems.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the limitations of existing deep - learning methods in signal - processing tasks, especially for one - dimensional signal - processing tasks (such as biomedical signal classification). Traditional deep - learning models (such as fully - connected networks and one - dimensional convolutional neural networks) have problems of high computational cost, difficult - to - interpret parameters, and the need for a large amount of labeled data. Specifically: 1. **High computational complexity**: Traditional deep neural networks (DNNs) and convolutional neural networks (CNNs) perform well in fields such as image processing, but in one - dimensional signal processing, their computational complexity is relatively high, especially during the training and inference stages. 2. **Poor parameter interpretability**: DNNs and CNNs are usually regarded as black - box models, and their parameters lack physical meaning and are difficult to interpret. This is especially disadvantageous for clinical applications that require interpretability. 3. **Large data requirements**: These models require a large amount of labeled data for training, and in many practical applications (such as telecommunications and biomedical engineering), it is difficult to obtain a large amount of labeled data. To solve these problems, the author introduced VPNet (Variable Projection Networks), which is a novel model - driven neural network architecture based on the Variable Projection (VP) theory. VPNet applies the VP operator to the neural network to achieve interpretable parameters, a compact network structure, and fast learning ability. Specifically, VPNet aims to: - **Improve computational efficiency**: Reduce the number of network parameters to lower the computational cost of training and inference. - **Enhance interpretability**: Make network parameters have physical meaning and be easy to interpret. - **Reduce dependence on a large amount of labeled data**: Use the VP operator to extract features and reduce the need for large - scale labeled data. In addition, the paper also shows the superior performance of VPNet in signal - processing tasks, especially its performance in classification, regression, and clustering problems is better than that of traditional fully - connected and one - dimensional convolutional networks.

VPNet: Variable Projection Networks

hvEEGNet: exploiting hierarchical VAEs on EEG data for neuroscience applications

Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE

vEEGNet: learning latent representations to reconstruct EEG raw data via variational autoencoders

An efficient and robust varying-parameter projection neural network for sparse signal reconstruction

D-VPnet: A Network for Real-time Dominant Vanishing Point Detection in Natural Scenes

VS-Net: Variable splitting network for accelerated parallel MRI reconstruction

MS-NetVLAD: Multi-Scale NetVLAD for Visual Place Recognition

LVP-net: A deep network of learning visual pathway for edge detection

Whole-brain Prediction Analysis with GraphNet

GaborNet Visual Encoding: A Lightweight Region-Based Visual Encoding Model With Good Expressiveness and Biological Interpretability

Variable Projection Algorithms: Theoretical Insights and A Novel Approach for Problems with Large Residual

TAVRNN: Temporal Attention-enhanced Variational Graph RNN Captures Neural Dynamics and Behavior

VPRTempo: A Fast Temporally Encoded Spiking Neural Network for Visual Place Recognition

Reliable Probabilistic Classification with Neural Networks

Random Projection in Deep Neural Networks

VC-PINN: Variable Coefficient Physical Information Neural Network For Forward And Inverse PDE Problems with Variable Coefficient

Efficient Scaling of Bayesian Neural Networks

Understanding Vector-Valued Neural Networks and Their Relationship with Real and Hypercomplex-Valued Neural Networks

FastVPINNs: Tensor-Driven Acceleration of VPINNs for Complex Geometries

Multi-modal Gaussian Process Variational Autoencoders for Neural and Behavioral Data