Abstract:Traditional neural networks employ fixed weights during inference, limiting their ability to adapt to changing input conditions, unlike biological neurons that adjust signal strength dynamically based on stimuli. This discrepancy between artificial and biological neurons constrains neural network flexibility and adaptability. To bridge this gap, we propose a novel framework for adaptive neural networks, where neuron weights are modeled as functions of the input signal, allowing the network to adjust dynamically in real-time. Importantly, we achieve this within the same traditional architecture of an Artificial Neural Network, maintaining structural familiarity while introducing dynamic adaptability. In our research, we apply Chebyshev polynomials as one of the many possible decomposition methods to achieve this adaptive weighting mechanism, with polynomial coefficients learned during training. Out of the 145 datasets tested, our adaptive Chebyshev neural network demonstrated a marked improvement over an equivalent MLP in approximately 8\% of cases, performing strictly better on 121 datasets. In the remaining 24 datasets, the performance of our algorithm matched that of the MLP, highlighting its ability to generalize standard neural network behavior while offering enhanced adaptability. As a generalized form of the MLP, this model seamlessly retains MLP performance where needed while extending its capabilities to achieve superior accuracy across a wide range of complex tasks. These results underscore the potential of adaptive neurons to enhance generalization, flexibility, and robustness in neural networks, particularly in applications with dynamic or non-linear data dependencies.

Beyond Weights Adaptation: a New Neuron Model with Trainable Activation Function and Its Supervised Learning.

A Neuron Model with Trainable Activation Function (TAF) and Its MFNN Supervised Learning

A New Kind of Neuron Model with A Tunable Activation Function and Its Applications

Non-linear Model Predictive Control Based on Neural Network Model with Modified Differential Evolution Adapting Weights

New perceptron model with tunable activation function

Adaptive Multi-Scale TF-net for High-Resolution Time-Frequency Representations

A neuron-weighted learning algorithm and its hardware implementation in associative memories

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Efficient Spiking Neural Networks with Sparse Selective Activation for Continual Learning

Tensor-based framework for training flexible neural networks

A New Artificial Neuron Proposal with Trainable Simultaneous Local and Global Activation Function

Activity-dependent Neuron Model for Noise Resistance

Deep Spiking Neural Network Using Spatio-temporal Backpropagation with Variable Resistance.

Flexible Transmitter Network

Effective Neural Network Training with a New Weighting Mechanism-Based Optimization Algorithm.

A survey on modern trainable activation functions

A Progressive Training Framework for Spiking Neural Networks with Learnable Multi-hierarchical Model

Bio-Inspired Adaptive Neurons for Dynamic Weighting in Artificial Neural Networks

Exploring the Relationship: Transformative Adaptive Activation Functions in Comparison to Other Activation Functions

Training Spiking Neural Networks with Accumulated Spiking Flow.

Trainable back-propagated functional transfer matrices