Abstract:We show that connectivity within the high-dimensional recurrent layer of a reservoir network is crucial for its performance. To this end, we systematically investigate the impact of network connectivity on its performance, i.e., we examine the symmetry and structure of the reservoir in relation to its computational power. Reservoirs with random and asymmetric connections are found to perform better for an exemplary Mackey-Glass time series than all structured reservoirs, including biologically inspired connectivities, such as small-world topologies. This result is quantified by the information processing capacity of the different network topologies which becomes highest for asymmetric and randomly connected networks.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **the influence of network connectivity on the performance of echo state networks, especially whether asymmetrically - connected networks can improve computational performance**. Specifically, through a systematic study of echo state networks with different topological structures (such as randomly - connected, small - world networks, etc.), the authors explored how the symmetry and structure of network connections affect their information - processing ability and computational performance. The study found that **echo state networks with random and asymmetric connections show better performance when processing nonlinear time - series tasks (such as Mackey - Glass time - series prediction)**, outperforming all structured networks, including biologically - inspired small - world topological structures. ### Key problems and solutions 1. **Research background and motivation**: - The performance of artificial neural networks (ANNs) is highly dependent on their connectivity, and this connectivity is usually tailored for specific tasks, making these networks very difficult to learn new tasks. - Biological neural networks (BNNs) have a decentralized and self - organizing learning process, but their structures are complex and difficult to implement. - Echo state networks (RC) combine the advantages of ANNs and BNNs, reducing training costs and improving computational efficiency. 2. **Research methods**: - Use the Mackey - Glass time - series as a benchmark test to evaluate the performance of different network topological structures. - Analyze five different network configurations: random - asymmetric (R - A), random - symmetric - asymmetric (RS - A), random - symmetric - symmetric (RS - S), Watts - Strogatz - asymmetric (WS - A) and Watts - Strogatz - symmetric (WS - S). - Quantify network performance through indicators such as mean - squared error (MSE) and effective prediction time (Tvp). 3. **Main conclusions**: - **Random networks with asymmetric connections (R - A) perform the best**, and their information - processing ability is significantly higher than other types of networks. - Networks with symmetric connections (such as WS - S and RS - S) require more capacity when processing nonlinear functions. - Information - processing - capacity (IPC) analysis shows that asymmetrically - connected networks can better handle complex nonlinear tasks. ### Formula summary - **Dynamics equation of echo state network**: \[ r_{m + 1}=(1-\varepsilon)r_m+\varepsilon\tanh (Wr_m + W_{\text{in}}u_{m + 1}) \] where \( r_m \) is the reservoir state at step \( m \), \( \varepsilon \) is the leakage rate, \( W \) is the reservoir matrix, \( W_{\text{in}} \) is the input weight matrix, and \( u_m \) is the input data. - **Information - processing - capacity (IPC)**: \[ C_M[R,\hat{u}]=1-\frac{\sum_{m = 1}^M(\hat{u}_m - u_p^{(m)})^2}{\sum_{m = 1}^M\hat{u}_m^2} \] where \( \hat{u}_m \) is the true value and \( u_p^{(m)} \) is the predicted value. - **Total information - processing - capacity (Total IPC)**: \[ \text{IPC}_{\text{total}}=\sum_{d = 1}^{\infty}\text{IPC}_d\approx\sum_{d = 1}^D\sum_{n = 1}^{N_d}C_M[R,\varphi_n(\xi_{-J}^m)] \] where \( D \) is the upper limit of the degree, \( J \) is the delay time, and \( N_d \) is the number of basis functions of degree \( d \). Through these studies, the authors demonstrated **the superiority of randomly - connected networks with asymmetric connections in computational performance**, for further optimization.

Asymmetrically connected reservoir networks learn better

The connectivity degree controls the difficulty in reservoir design of random boolean networks

The connectivity degree controls the difficulty of RBN reservoir design

Decoding Neuronal Networks: A Reservoir Computing Approach for Predicting Connectivity and Functionality

Computational Differences Between Asymmetrical and Symmetrical Networks

An Online Structural Plasticity Rule for Generating Better Reservoirs

Separation capacity of linear reservoirs with random connectivity matrix

Nonlinear Neural Dynamics and Classification Accuracy in Reservoir Computing

Reservoir Computing with a Small-World Network for Discriminating Two Sequential Stimuli.

Effect of dilution in asymmetric recurrent neural networks

Reservoir Computing meets Recurrent Kernels and Structured Transforms

Connectivity structure and dynamics of nonlinear recurrent neural networks

The Impact of Network Connectivity on Collective Learning

Excitatory/Inhibitory Balance Emerges as a Key Factor for RBN Performance, Overriding Attractor Dynamics

Spatiotemporal dynamics and reliable computations in recurrent spiking neural networks

Hierarchical architectures in reservoir computing systems

Investigating Biologically Plausible Neural Networks for Reservoir Computing Solutions

Reservoir computing dissection and visualization based on directed network embedding

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

Seeing double with a multifunctional reservoir computer

Associative memory on small-world networks