Abstract:Existing learning methods often struggle to balance interpretability and predictive performance. While models like nearest neighbors and non-negative matrix factorization (NMF) offer high interpretability, their predictive performance on supervised learning tasks is often limited. In contrast, neural networks based on the multi-layer perceptron (MLP) support the modular construction of expressive architectures and tend to have better recognition accuracy but are often regarded as black boxes in terms of interpretability. Our approach aims to strike a better balance between these two aspects through the use of a building block based on NMF that incorporates supervised neural network training methods to achieve high predictive performance while retaining the desirable interpretability properties of NMF. We evaluate our Predictive Factorized Coupling (PFC) block on small datasets and show that it achieves competitive predictive performance with MLPs while also offering improved interpretability. We demonstrate the benefits of this approach in various scenarios, such as continual learning, training on non-i.i.d. data, and knowledge removal after training. Additionally, we show examples of using the PFC block to build more expressive architectures, including a fully-connected residual network as well as a factorized recurrent neural network (RNN) that performs competitively with vanilla RNNs while providing improved interpretability. The PFC block uses an iterative inference algorithm that converges to a fixed point, making it possible to trade off accuracy vs computation after training but also currently preventing its use as a general MLP replacement in some scenarios such as training on very large datasets. We provide source code at <a class="link-external link-https" href="https://github.com/bkvogel/pfc" rel="external noopener nofollow">this https URL</a>

Analytic Continuation by Feature Learning

Noise Enhanced Neural Networks for Analytic Continuation

Neural Network Analytic Continuation for Monte Carlo: Improvement by Statistical Errors

Neural Network Analytic Continuation for Monte Carlo: Improvement by Statistical Errors

Analytic continuation via domain-knowledge free machine learning

Uncertainty Estimation With Neural Processes for Meta-Continual Learning

Autoencoder-based analytic continuation method for strongly correlated quantum systems

FL-Net: A multi-scale cross-decomposition network with frequency external attention for long-term time series forecasting

Training biases in machine learning for the analytic continuation of quantum many-body Green's functions

Feature Expansion for Graph Neural Networks

FL-GNNs: Robust Network Representation via Feature Learning guided Graph Neural Networks

A spectral method for a Fokker-Planck equation in neuroscience with applications in neural networks with learning rules

FAN: Fourier Analysis Networks

GLFNet: Global and Local Frequency-domain Network for Long-term Time Series Forecasting

FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysis

Continuous Dynamic-NeRF: Spline-NeRF

Feature Forgetting in Continual Representation Learning

A Spectral Condition for Feature Learning

FSW-GNN: A Bi-Lipschitz WL-Equivalent Graph Neural Network

Continuous Spectral Reconstruction from RGB Images via Implicit Neural Representation

An NMF-Based Building Block for Interpretable Neural Networks With Continual Learning