Abstract:Insects, such as fruit flies and honey bees, can solve simple associative learning tasks and learn abstract concepts such as "sameness" and "difference", which is viewed as a higher-order cognitive function and typically thought to depend on top-down neocortical processing. Empirical research with fruit flies strongly supports that a randomized representational architecture is used in olfactory processing in insect brains. Based on these results, we propose a Randomly Weighted Feature Network (RWFN) that incorporates randomly drawn, untrained weights in an encoder that uses an adapted linear model as a decoder. The randomized projections between input neurons and higher-order processing centers in the input brain is mimicked in RWFN by a single-hidden-layer neural network that specially structures latent representations in the hidden layer using random Fourier features that better represent complex relationships between inputs using kernel approximation. Because of this special representation, RWFNs can effectively learn the degree of relationship among inputs by training only a linear decoder model. We compare the performance of RWFNs to LTNs for Semantic Image Interpretation (SII) tasks that have been used as a representative example of how LTNs utilize reasoning over first-order logic to surpass the performance of solely data-driven methods. We demonstrate that compared to LTNs, RWFNs can achieve better or similar performance for both object classification and detection of the part-of relations between objects in SII tasks while using much far fewer learnable parameters (1:62 ratio) and a faster learning process (1:2 ratio of running speed). Furthermore, we show that because the randomized weights do not depend on the data, several decoders can share a single randomized encoder, giving RWFNs a unique economy of spatial scale for simultaneous classification tasks.

An Interpretable Constructive Algorithm for Incremental Random Weight Neural Networks and Its Application.

A Compact Constraint Incremental Method for Random Weight Networks and Its Application

Interpretable Neural Networks with Random Constructive Algorithm

Adaptive inertia weights: an effective way to improve parameter estimation of hidden layer in stochastic configuration networks

A Gaussian Mixture Framework for Incremental Nonparametric Regression with Topology Learning Neural Networks

An Iterative Learning Algorithm for Feedforward Neural Networks with Random Weights

Rethinking the Usage of Batch Normalization and Dropout in the Training of Deep Neural Networks

Multitarget Incremental Modeling Via Random Weight Networks

Building Feedforward Neural Networks with Random Weights for Large Scale Datasets

ICFD: an Incremental Learning Method Based on Data Feature Distribution.

HyperINR: A Fast and Predictive Hypernetwork for Implicit Neural Representations via Knowledge Distillation

A Gradient Free Neural Network Framework Based on Universal Approximation Theorem

Incremental Learning Paradigm with Privileged Information for Random Vector Functional-Link Networks: IRVFL+

Adaptive Incremental Learning Stochastic Configuration Network and Its NIR Modeling Application

On How Iterative Magnitude Pruning Discovers Local Receptive Fields in Fully Connected Neural Networks

IF2Net: Innately Forgetting-Free Networks for Continual Learning

An Insect-Inspired Randomly, Weighted Neural Network with Random Fourier Features For Neuro-Symbolic Relational Learning

INA: Incremental Network Approximation Algorithm for Limited Precision Deep Neural Networks

Compact Incremental Random Weight Network for Estimating the Underground Airflow Quantity

Incremental Online Learning of Randomized Neural Network with Forward Regularization

Incremental Learning Without Looking Back: a Neural Connection Relocation Approach