Abstract:Neural networks, as powerful tools for data mining and knowledge engineering, can learn from data to build feature‐based classifiers and nonlinear predictive models. Training neural networks involves the optimization of nonconvex objective functions, and usually, the learning process is costly and infeasible for applications associated with data streams. A possible, albeit counterintuitive, alternative is to randomly assign a subset of the networks’ weights so that the resulting optimization task can be formulated as a linear least‐squares problem. This methodology can be applied to both feedforward and recurrent networks, and similar techniques can be used to approximate kernel functions. Many experimental results indicate that such randomized models can reach sound performance compared to fully adaptable ones, with a number of favorable benefits, including (1) simplicity of implementation, (2) faster learning with less intervention from human beings, and (3) possibility of leveraging overall linear regression and classification algorithms (e.g., ℓ 1 norm minimization for obtaining sparse formulations). This class of neural networks attractive and valuable to the data mining community, particularly for handling large scale data mining in real‐time. However, the literature in the field is extremely vast and fragmented, with many results being reintroduced multiple times under different names. This overview aims to provide a self‐contained, uniform introduction to the different ways in which randomization can be applied to the design of neural networks and kernel functions. A clear exposition of the basic framework underlying all these approaches helps to clarify innovative lines of research, open problems, and most importantly, foster the exchanges of well‐known results throughout different communities. WIREs Data Mining Knowl Discov 2017, 7:e1200. doi: 10.1002/widm.1200This article is categorized under: Technologies > Machine Learning

Random Neural Graph Generation with Structure Evolution.

Evolution beats random chance: Performance-dependent network evolution for enhanced computational capacity

Biologically Inspired Structure Learning with Reverse Knowledge Distillation for Spiking Neural Networks

Exploring Randomly Wired Neural Networks for Image Recognition

EvoNet: A Neural Network for Predicting the Evolution of Dynamic Graphs

Genetic-GNN: Evolutionary architecture search for Graph Neural Networks

A Neural Collapse Perspective on Feature Evolution in Graph Neural Networks

A Novel Local-World Evolving Network Model for Power Grid

Emergence of brain-inspired small-world spiking neural network through neuroevolution

Universal structural patterns in sparse recurrent neural networks

Optimization Algorithm Inspired Deep Neural Network Structure Design

Structure of Artificial Neural Networks -- Empirical Investigations

Research of Complex Network Dynamics Evolution

A Deep Graph Neural Networks Architecture Design: From Global Pyramid-like Shrinkage Skeleton to Local Topology Link Rewiring

GraphPNAS: Learning Distribution of Good Neural Architectures via Deep Graph Generative Models

A Deep Graph Neural Networks Architecture Design: From Global Pyramid-like Shrinkage Skeleton to Local Link Rewiring

An Artificial Neural Network Functionalized by Evolution

Understanding Patterns of Deep Learning ModelEvolution in Network Architecture Search

Emergent Structures and Lifetime Structure Evolution in Artificial Neural Networks

Optimising the topology of complex neural networks

Randomness in Neural Networks: an Overview