Abstract:figure legend Neural networks execute the transformation of inputs into the neural activity that enables the execution of a task. Thus, they have the potential to establish causal relationships between the input, the elements of a circuit, the network activity, and eventually behavior. When the purpose of building a model is to understand the mechanisms that enable behavior in biological circuits, the design of network models should be guided by three principles: (1) biological constraints are necessary (e.g., connectivity), (2) the network activity should match the recorded activity, and (3) the network should be able to perform the task of interest. Machine‐learning‐based training can help accomplish these goals. Due to the staggering complexity of the brain and its neural circuitry, neuroscientists rely on the analysis of mathematical models to elucidate its function. From Hodgkin and Huxley's detailed description of the action potential in 1952 to today, new theories and increasing computational power have opened up novel avenues to study how neural circuits implement the computations that underlie behavior. Computational neuroscientists have developed many models of neural circuits that differ in complexity, biological realism, or emergent network properties. With recent advances in experimental techniques for detailed anatomical reconstructions of large‐scale activity recordings, precise biological data has become more available. The challenge when building network models is to reflect experimental results, either through a high level of detail or by finding an appropriate level of abstraction. Meanwhile, machine learning has facilitated the development of artificial neural networks (ANN), which are trained to perform specific tasks. While they have proven successful at achieving task‐oriented behavior, they are often abstract constructs that differ in many features from the physiology of brain circuits. Thus, it is unclear whether the mechanisms underlying computation in biological circuits can be investigated by analyzing artificial networks that accomplish the same function but differ in their mechanisms. Here, we argue that building biologically realistic network models is crucial to establishing causal relationships between neurons, synapses, circuits, and behavior. More specifically, we advocate for network models that consider the connectivity structure and the recorded activity dynamics while evaluating task performance.

Interpretability of Neural Network With Physiological Mechanisms

Multilevel Interpretability Of Artificial Neural Networks: Leveraging Framework And Methods From Neuroscience

Neural Networks from Biological to Artificial and Vice Versa

How to incorporate biological insights into network models and why it matters

Functional Network: A Novel Framework for Interpretability of Deep Neural Networks

A Survey of the Interpretability Aspect of Deep Learning Models

Interpretability of deep learning models: A survey of results

On Interpretability of Artificial Neural Networks: A Survey

From Neurons to Neutrons: A Case Study in Interpretability

What is Interpretability?

Opening the Black Box of Neural Networks: Methods for Interpreting Neural Network Models in Clinical Applications

Novel Interpretable Mechanism of Neural Networks Based on Network Decoupling Method

Interpretability of Machine Learning Methods Applied to Neuroimaging

Looking deeper into interpretable deep learning in neuroimaging: a comprehensive survey

From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction

A Survey on Neural Network Interpretability

Interpretability for Reliable, Efficient, and Self-Cognitive DNNs: from Theories to Applications.

Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations

Biophysical models of cis-regulation as interpretable neural networks

Perspectives and constraints on neural network models of neurobiological processes