Abstract:Understanding the real-time dynamical mechanisms of neural systems remains a significant issue, preventing the development of efficient neural technology and user trust. This is because the mechanisms, involving various neural spatial-temporal ingredients [i.e., neural structure (NS), neural dynamics (ND), neural plasticity (NP), and neural memory (NM)], are too complex to interpret and analyze altogether. While advanced tools have been developed using explainable artificial intelligence (XAI), node-link diagram, topography map, and other visualization techniques, they still fail to monitor and visualize all of these neural ingredients online. Accordingly, we propose here for the first time "NeuroVis," real-time neural spatial-temporal information measurement and visualization, as a method/tool to measure temporal neural activities and their propagation throughout the network. By using this neural information along with the connection strength and plasticity, NeuroVis can visualize the NS, ND, NM, and NP via i) spatial 2D position and connection, ii) temporal color gradient, iii) connection thickness, and iv) temporal luminous intensity and change of connection thickness, respectively. This study presents three use cases of NeuroVis to evaluate its performance: i) function approximation using a modular neural network with recurrent and feedforward topologies together with supervised learning, ii) robot locomotion control and learning using the same modular network with reinforcement learning, and iii) robot locomotion control and adaptation using another larger-scale adaptive modular neural network. The use cases demonstrate how NeuroVis tracks and analyzes all neural ingredients of various (embodied) neural systems in real-time under the robot operating system (ROS) framework. To this end, it will offer the opportunity to better understand embodied dynamic neural information processes, boost efficient neural technology development, and enhance user trust.

NeuroView-RNN: It's About Time

NeuroView: Explainable Deep Network Decision Making

Residual Recurrent Neural Networks for Learning Sequential Representations.

Towards Interpreting Recurrent Neural Networks Through Probabilistic Abstraction

Interpretable Additive Recurrent Neural Networks For Multivariate Clinical Time Series

A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts

Neural Reasoning Networks: Efficient Interpretable Neural Networks With Automatic Textual Explanations

Understanding Hidden Memories of Recurrent Neural Networks

BrainNNExplainer: An Interpretable Graph Neural Network Framework for Brain Network based Disease Analysis

Deep RNN Framework for Visual Sequential Applications

Investigating Explainability Methods in Recurrent Neural Network Architectures for Financial Time Series Data

Exploring weight initialization, diversity of solutions, and degradation in recurrent neural networks trained for temporal and decision-making tasks

RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records

NeuroVis: Real-Time Neural Information Measurement and Visualization of Embodied Neural Systems

Input-Cell Attention Reduces Vanishing Saliency of Recurrent Neural Networks

Inferring stochastic low-rank recurrent neural networks from neural data

MV-RNN: A Multi-View Recurrent Neural Network for Sequential Recommendation

From CNN to CNN + RNN: Adapting Visualization Techniques for Time-Series Anomaly Detection

RWKV-TS: Beyond Traditional Recurrent Neural Network for Time Series Tasks

Neural network interpretability with layer-wise relevance propagation: novel techniques for neuron selection and visualization