Abstract:Artificial Neural Networks (ANNs) have gained widespread applications across various areas in recent years. The ANN design was initially inspired by the principle of biology. The biological neural network's fundamental response process comprises information transmission and aggregation. The information transmission in biological neurons is often achieved by triggering action potentials that propagate through axons. ANNs utilize activation mechanisms to simulate such biological behavior. However, previous studies have only considered static response conditions, while the biological neuron's response conditions are typically dynamic, depending on multiple factors such as neuron properties and the real-time environment. Therefore, the dynamic response conditions of biological neurons could help improve the static ones of existing activations in ANNs. Additionally, the biological neuron's aggregated response exhibits high specificity for different categories, allowing the nervous system to differentiate and identify objects. Inspired by these biological patterns, we propose a novel Dynamic Neural Response Tuning (DNRT) mechanism, which aligns the response patterns of ANNs with those of biological neurons. DNRT comprises Response-Adaptive Activation (RAA) and Aggregated Response Regularization (ARR), mimicking the biological neuron's information transmission and aggregation behaviors. RAA dynamically adjusts the response condition based on the strength and characteristics of the input signal. ARR is devised to enhance the network's ability to learn category specificity by imposing constraints on the network's response distribution. Extensive experimental studies indicate that the proposed DNRT is highly interpretable, applicable to various mainstream network architectures, and can achieve remarkable performance compared with existing neural response mechanisms in multiple tasks and domains.

A New Kind of Neuron Model with A Tunable Activation Function and Its Applications

A Neuron Model with Trainable Activation Function (TAF) and Its MFNN Supervised Learning

Beyond Weights Adaptation: a New Neuron Model with Trainable Activation Function and Its Supervised Learning.

New perceptron model with tunable activation function

Adaptive Multi-Scale TF-net for High-Resolution Time-Frequency Representations

Efficient Spiking Neural Networks with Sparse Selective Activation for Continual Learning

Exploring the Relationship: Transformative Adaptive Activation Functions in Comparison to Other Activation Functions

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Deep Spiking Neural Network Using Spatio-temporal Backpropagation with Variable Resistance.

A New Neural Network Model And Its Application

A New Artificial Neuron Proposal with Trainable Simultaneous Local and Global Activation Function

No One-Size-Fits-All Neurons: Task-based Neurons for Artificial Neural Networks

Tensor-based framework for training flexible neural networks

Dynamic Neural Response Tuning

Training Stronger Spiking Neural Networks with Biomimetic Adaptive Internal Association Neurons

An Ann Modeling Method With Optimizing Net Structure And Activation Function Types

Circuit Design of an Adjustable Neuron Activation Function and Its Derivative

Compact Modeling of Advanced Gate-All-Around Nanosheet FETs Using Artificial Neural Network

A New ANN-SNN Conversion Method with High Accuracy, Low Latency and Good Robustness

Twin Network Augmentation: A Novel Training Strategy for Improved Spiking Neural Networks and Efficient Weight Quantization

A Method on Searching Better Activation Functions