AI without networks

Partha Pratim Mitra,Clément Sire

DOI: https://doi.org/10.1101/2023.12.09.570922

2024-06-05

Abstract:Contemporary Artificial Intelligence (AI) stands on two legs: large training data corpora and many-parameter artificial neural networks (ANNs). The data corpora are needed to represent the complexity and heterogeneity of the world. The role of the networks is less transparent due to the obscure dependence of the network parameters and outputs on the training data and inputs. This raises problems, ranging from technical-scientific to legal-ethical. We hypothesize that a transparent approach to machine learning is possible without using networks at all. By generalizing a parameter-free, statistically consistent data interpolation method, which we analyze theoretically in detail, we develop a network-free framework for AI incorporating generative modeling. We demonstrate this framework with examples from three different disciplines - ethology, control theory, and mathematics. Our generative Hilbert framework applied to the trajectories of small groups of swimming fish outperformed state-of-the-art traditional mathematical behavioral models and current ANN-based models. We demonstrate pure data interpolation based control by stabilizing an inverted pendulum and a driven logistic map around unstable fixed points. Finally, we present a mathematical application by predicting zeros of the Riemann Zeta function, achieving comparable performance as a transformer network. We do not suggest that the proposed framework will always outperform networks as over-parameterized networks can interpolate. However, our framework is theoretically sound, transparent, deterministic, and parameter free: remarkably, it does not require any compute-expensive training, does not involve optimization, has no model selection, and is easily reproduced and ported. We also propose an easily computed method of credit assignment based on this framework, to help address ethical-legal challenges raised by generative AI.

Neuroscience

What problem does this paper attempt to address?

This paper proposes a network-free approach for artificial intelligence (AI) that aims to address the issues of low transparency, poor interpretability, and legal ethics problems caused by the reliance on large amounts of training data and complex neural networks in current AI technologies. The authors assume that it is possible to develop a transparent machine learning framework without using a network by extending a parameter-free, statistically consistent data interpolation method - Hilbert kernel interpolator for generative modeling. The paper demonstrates the application of this framework in examples from different disciplines such as animal behavior, control theory, and mathematics, including modeling of fish school behavior, inverted pendulum control, and Riemann zeta function zero prediction, showing better performance than traditional mathematical models and neural network-based methods.

The main contributions of the paper include:
1. Proposing a parameter-free, statistically consistent interpolation method based on the Hilbert kernel, which does not require expensive computational training, optimization, model selection, and is easy to reproduce and transfer.
2. Providing a simple calculation method for credit assignment to address the ethical and legal challenges brought by generative AI.
3. Demonstrating good performance in some applications but emphasizing that performance is not the sole evaluation criterion, especially in scientific and biomedical applications, where understandability, transparency, and theoretical foundations are equally important.

The paper points out that although over-parameterized networks may perform well in certain tasks, the Hilbert framework offers theoretical clarity, addressing the issues of network parameter interpretation and output understanding, and helps improve replicability and portability.