Proceedings of AAAI 2019 Workshop on Network Interpretability for Deep Learning

Quanshi Zhang,Lixin Fan,Bolei Zhou

DOI: https://doi.org/10.48550/arXiv.1901.08813

2020-07-29

Abstract:This is the Proceedings of AAAI 2019 Workshop on Network Interpretability for Deep Learning

Artificial Intelligence

What problem does this paper attempt to address?

Visual Interpretability forDeepLearning

Quanshi Zhang,Song-Chun Zhu

2018-01-01

Abstract:This paper reviews recent studies in emerging directions of understanding neural-network representations and learning neural networks with interpretable/disentangled middle-layer representations. Although deep neural networks have exhibited superior performance in various tasks, the interpretability is always an Achilles' heel of deep neural networks. At present, deep neural networks obtain a high discrimination power at the cost of low interpretability of their black-box representations. We believe that the high model interpretability may help people to break several bottlenecks of deep learning, e.g. learning from very few annotations, learning via human-computer communications at the semantic level, and semantically debugging network representations. In this paper, we focus on convolutional neural networks (CNNs), and we revisit the visualization of CNN representations, methods of diagnosing representations of pre-trained CNNs, approaches for disentangling pre-trained CNN representations, learning of CNNs with disentangled representations, and middle-to-end learning based on model interpretability. Finally, we discuss prospective trends of explainable arti cial intelligence.
A Survey on Neural Network Interpretability

Yu Zhang,Peter Tio,Ale Leonardis,Ke Tang,Peter Tino,Ales Leonardis

DOI: https://doi.org/10.1109/tetci.2021.3100641

2021-10-01

IEEE Transactions on Emerging Topics in Computational Intelligence

Abstract:Along with the great success of deep neural networks, there is also growing concern about their black-box nature. The interpretability issue affects people's trust on deep learning systems. It is also related to many ethical problems, e.g., algorithmic discrimination. Moreover, interpretability is a desired property for deep networks to become powerful tools in other research fields, e.g., drug discovery and genomics. In this survey, we conduct a comprehensive review of the neural network interpretability research. We first clarify the definition of interpretability as it has been used in many different contexts. Then we elaborate on the importance of interpretability and propose a novel taxonomy organized along three dimensions: type of engagement (passive vs. active interpretation approaches), the type of explanation, and the focus (from local to global interpretability). This taxonomy provides a meaningful 3D view of distribution of papers from the relevant literature as two of the dimensions are not simply categorical but allow ordinal subcategories. Finally, we summarize the existing interpretability evaluation methods and suggest possible research directions inspired by our new taxonomy.
Visual Interpretability for Deep Learning: a Survey

Quanshi Zhang,Song-Chun Zhu

DOI: https://doi.org/10.48550/arXiv.1802.00614

2018-02-02

Computer Vision and Pattern Recognition

Abstract:This paper reviews recent studies in understanding neural-network representations and learning neural networks with interpretable/disentangled middle-layer representations. Although deep neural networks have exhibited superior performance in various tasks, the interpretability is always the Achilles' heel of deep neural networks. At present, deep neural networks obtain high discrimination power at the cost of low interpretability of their black-box representations. We believe that high model interpretability may help people to break several bottlenecks of deep learning, e.g., learning from very few annotations, learning via human-computer communications at the semantic level, and semantically debugging network representations. We focus on convolutional neural networks (CNNs), and we revisit the visualization of CNN representations, methods of diagnosing representations of pre-trained CNNs, approaches for disentangling pre-trained CNN representations, learning of CNNs with disentangled representations, and middle-to-end learning based on model interpretability. Finally, we discuss prospective trends in explainable artificial intelligence.
Functional Network: A Novel Framework for Interpretability of Deep Neural Networks

Ben Zhang,Zhetong Dong,Junsong Zhang,Hongwei Lin

DOI: https://doi.org/10.1016/j.neucom.2022.11.035

IF: 6

2023-01-01

Neurocomputing

Abstract:The layered structure of deep neural networks hinders the use of numerous analysis tools and thus the development of its interpretability. Inspired by the success of functional brain networks, we propose a novel framework for interpretability of deep neural networks, that is, the functional network. We con-struct the functional network of fully connected networks and explore its small-worldness. In our exper-iments, the mechanisms of regularization methods, namely, batch normalization and dropout, are revealed using graph theoretical analysis and topological data analysis. Our empirical analysis shows the following: (1) batch normalization enhances model performance by increasing the global efficiency and the number of loops but reduces adversarial robustness by lowering the fault tolerance. (2) Dropout improves generalization and robustness of models by improving the functional specialization and fault tolerance. (3) The models with different regularizations can be clustered accurately according to their functional topological differences, reflecting the great potential of the functional network and topological data analysis in interpretability.(c) 2022 Elsevier B.V. All rights reserved.
A Comprehensive Review of Deep Neural Network Interpretation Using Topological Data Analysis

Ben Zhang,Zitong He,Hongwei Lin

DOI: https://doi.org/10.1016/j.neucom.2024.128513

IF: 6

2024-01-01

Neurocomputing

Abstract:Deep neural networks have achieved significant success across various fields, but their intrinsic black-box nature hinders the further development. Addressing the interpretability challenges, topological data analysis has emerged as a promising tool to reveal these complex models. In this work, we present a review of the emerging field of interpreting deep neural networks using topological data analysis. We organize the existing body of work into distinct analytical categories, highlighting interpretations based on the topology of data, network structural characteristics, network functional characteristics, and techniques derived from Mapper. The objective of this paper is to extract the research pattern of this area, and point out the future research direction.
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond

Xuhong Li,Haoyi Xiong,Xingjian Li,Xuanyu Wu,Xiao Zhang,Ji Liu,Jiang Bian,Dejing Dou

DOI: https://doi.org/10.1007/s10115-022-01756-8

IF: 2.7

2022-09-15

Knowledge and Information Systems

Abstract:Deep neural networks have been well-known for their superb handling of various machine learning and artificial intelligence tasks. However, due to their over-parameterized black-box nature, it is often difficult to understand the prediction results of deep models. In recent years, many interpretation tools have been proposed to explain or reveal how deep models make decisions. In this paper, we review this line of research and try to make a comprehensive survey. Specifically, we first introduce and clarify two basic concepts—interpretations and interpretability—that people usually get confused about. To address the research efforts in interpretations, we elaborate the designs of a number of interpretation algorithms, from different perspectives, by proposing a new taxonomy. Then, to understand the interpretation results, we also survey the performance metrics for evaluating interpretation algorithms. Further, we summarize the current works in evaluating models' interpretability using "trustworthy" interpretation algorithms. Finally, we review and discuss the connections between deep models' interpretations and other factors, such as adversarial robustness and learning from interpretations, and we introduce several open-source libraries for interpretation algorithms and evaluation approaches.

computer science, information systems, artificial intelligence
Demystifying Deep Learning in Networking

Ying Zheng,Ziyu Liu,Xinyu You,Yuedong Xu,Junchen Jiang

DOI: https://doi.org/10.1145/3232565.3232569

2018-01-01

Abstract:We are witnessing a surge of efforts in networking community to develop deep neural networks (DNNs) based approaches to networking problems. Most results so far have been remarkably promising, which is arguably surprising given how intensively these problems have been studied before. Despite these promises, there has not been much systematic work to understand the inner workings of these DNNs trained in networking settings, their generalizability in different workloads, and their potential synergy with domain-specific knowledge. The problem of model opacity would eventually impede the adoption of DNN-based solutions in practice. This position paper marks the first attempt to shed light on the interpretability of DNNs used in networking problems. Inspired by recent research in ML towards interpretable ML models, we call upon this community to similarly develop techniques and leverage domain-specific insights to demystify the DNNs trained in networking settings, and ultimately unleash the potential of DNNs in an explainable and reliable way.
On Interpretability of Artificial Neural Networks: A Survey

Feng-Lei Fan,Jinjun Xiong,Mengzhou Li,Ge Wang

DOI: https://doi.org/10.1109/trpms.2021.3066428

2021-11-01

IEEE Transactions on Radiation and Plasma Medical Sciences

Abstract:Deep learning as represented by the artificial deep neural networks (DNNs) has achieved great success recently in many important areas that deal with text, images, videos, graphs, and so on. However, the black-box nature of DNNs has become one of the primary obstacles for their wide adoption in mission-critical applications such as medical diagnosis and therapy. Because of the huge potentials of deep learning, increasing the interpretability of deep neural networks has recently attracted much research attention. In this paper, we propose a simple but comprehensive taxonomy for interpretability, systematically review recent studies in improving interpretability of neural networks, describe applications of interpretability in medicine, and discuss possible future research directions of interpretability, such as in relation to fuzzy logic and brain science.
A Survey of the Interpretability Aspect of Deep Learning Models

Eliot Spitzer,Rona Miles

DOI: https://doi.org/10.53759/0088/jbsha202303006

2023-01-05

Journal of Biomedical and Sustainable Healthcare Applications

Abstract:Deep neural networks have attained near-human degree of quality in images, textual, audio, and video recording categorization and predictions tasks. The networks, on the other hand, are still typically thought of as black-box functional probabilistic models that transfer an input data to a trained classifier. Integrating these systems into mission-critical activities like clinical diagnosis, scheduling, and management is the next stage in this human-machine evolutionary change, and it necessitates a degree of confidence in the technology output. Statistical measures are often employed to estimate an output's volatility. The idea of trust, on the other hand, is dependent on a human's sight into a machine's inner workings. To put it another way, the neural networks must justify its outputs in a way that is intelligible to humans, leading to new insights into its internal workings. "Interpretable deep networks" is the name we give to such networks. The concept of interpretability is not one-dimensional. Indeed, the variability of an interpretation owing to varying degrees of human comprehension necessitates the existence of a plethora of characteristics that together define interpretability. Furthermore, the model's interpretations may be expressed in terms of low-level network variables or input properties. We describe several of the variables that are helpful for model interpretability in this study, as well as previous work on those dimensions. We do a gap analysis to determine what remains to be improved to increase models interpretability as step of the procedure.
Interpreting AI for Networking: Where We Are and Where We Are Going

Tianzhu Zhang,Han Qiu,Marco Mellia,Yuanjie Li,Hewu Li,Ke Xu

DOI: https://doi.org/10.1109/mcom.001.2100736

IF: 9.03

2022-02-01

IEEE Communications Magazine

Abstract:In recent years, artificial intelligence (AI) techniques have been increasingly adopted to tackle networking problems. Although AI algorithms can deliver high-quality solutions, most of them are inherently intricate and erratic for human cognition. This lack of interpretability tremendously hinders the commercial success of AI-based solutions in practice. To cope with this challenge, networking researchers are starting to explore explainable AI (XAI) techniques to make AI models interpretable, manageable, and trustworthy. In this article, we overview the application of AI in networking and discuss the necessity for interpretability. Next, we review the current research on interpreting AI-based networking solutions and systems. At last, we envision future challenges and directions. The ultimate goal of this article is to present a general guideline for AI and networking practitioners and motivate the continuous advancement of AI-based solutions in modern communication networks.

telecommunications,engineering, electrical & electronic
Feature Analysis Network: An Interpretable Idea in Deep Learning

Xinyu Li,Xiaoguang Gao,Qianglong Wang,Chenfeng Wang,Bo Li,Kaifang Wan

DOI: https://doi.org/10.1007/s12559-023-10238-0

IF: 4.89

2024-01-21

Cognitive Computation

Abstract:Deep Learning (DL) stands out as a leading model for processing high-dimensional data, where the nonlinear transformation of hidden layers effectively extracts features. However, these unexplainable features make DL a low interpretability model. Conversely, Bayesian network (BN) is transparent and highly interpretable, and it can be helpful for interpreting DL. To improve the interpretability of DL from the perspective of feature cognition, we propose the feature analysis network (FAN), a DL structure fused with BN. FAN retains the DL feature extraction capability and applies BN as the output layer to learn the relationships between the features and the outputs. These relationships can be probabilistically represented by the structure and parameters of the BN, intuitively. In a further study, a correlation clustering-based feature analysis network (cc-FAN) is proposed to detect the correlations among inputs and to preserve this information to explain the features' physical meaning to a certain extent. To quantitatively evaluate the interpretability of the model, we design the network simplification and interpretability indicators separately. Experiments on eight datasets show that FAN has better interpretability than that of the other models with basically unchanged model accuracy and similar model complexities. On the radar effect mechanism dataset, from the feature structure-based relevance interpretability indicator, FAN is up to 4.8 times better than that of the other models, and cc-FAN is up to 21.5 times better than that of the other models. FAN and cc-FAN enhance the interpretability of the DL model structure from the aspects of features; moreover, based on the input correlations, cc-FAN can help us to better understand the physical meaning of features.

computer science, artificial intelligence,neurosciences
Interpretable neural networks: principles and applications

Zhuoyang Liu,Feng Xu

DOI: https://doi.org/10.3389/frai.2023.974295

2023-10-13

Abstract:In recent years, with the rapid development of deep learning technology, great progress has been made in computer vision, image recognition, pattern recognition, and speech signal processing. However, due to the black-box nature of deep neural networks (DNNs), one cannot explain the parameters in the deep network and why it can perfectly perform the assigned tasks. The interpretability of neural networks has now become a research hotspot in the field of deep learning. It covers a wide range of topics in speech and text signal processing, image processing, differential equation solving, and other fields. There are subtle differences in the definition of interpretability in different fields. This paper divides interpretable neural network (INN) methods into the following two directions: model decomposition neural networks, and semantic INNs. The former mainly constructs an INN by converting the analytical model of a conventional method into different layers of neural networks and combining the interpretability of the conventional model-based method with the powerful learning capability of the neural network. This type of INNs is further classified into different subtypes depending on which type of models they are derived from, i.e., mathematical models, physical models, and other models. The second type is the interpretable network with visual semantic information for user understanding. Its basic idea is to use the visualization of the whole or partial network structure to assign semantic information to the network structure, which further includes convolutional layer output visualization, decision tree extraction, semantic graph, etc. This type of method mainly uses human visual logic to explain the structure of a black-box neural network. So it is a post-network-design method that tries to assign interpretability to a black-box network structure afterward, as opposed to the pre-network-design method of model-based INNs, which designs interpretable network structure beforehand. This paper reviews recent progress in these areas as well as various application scenarios of INNs and discusses existing problems and future development directions.
Proceedings of ICML 2021 Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI

Quanshi Zhang,Tian Han,Lixin Fan,Zhanxing Zhu,Hang Su,Ying Nian Wu,Jie Ren,Hao Zhang

DOI: https://doi.org/10.48550/arXiv.2107.08821

2021-07-26

Abstract:This is the Proceedings of ICML 2021 Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI. Deep neural networks (DNNs) have undoubtedly brought great success to a wide range of applications in computer vision, computational linguistics, and AI. However, foundational principles underlying the DNNs' success and their resilience to adversarial attacks are still largely missing. Interpreting and theorizing the internal mechanisms of DNNs becomes a compelling yet controversial topic. This workshop pays a special interest in theoretic foundations, limitations, and new application trends in the scope of XAI. These issues reflect new bottlenecks in the future development of XAI.

Machine Learning,Artificial Intelligence,Computers and Society
Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks

Tilman Räuker,Anson Ho,Stephen Casper,Dylan Hadfield-Menell

2023-08-19

Abstract:The last decade of machine learning has seen drastic increases in scale and capabilities. Deep neural networks (DNNs) are increasingly being deployed in the real world. However, they are difficult to analyze, raising concerns about using them without a rigorous understanding of how they function. Effective tools for interpreting them will be important for building more trustworthy AI by helping to identify problems, fix bugs, and improve basic understanding. In particular, "inner" interpretability techniques, which focus on explaining the internal components of DNNs, are well-suited for developing a mechanistic understanding, guiding manual modifications, and reverse engineering solutions. Much recent work has focused on DNN interpretability, and rapid progress has thus far made a thorough systematization of methods difficult. In this survey, we review over 300 works with a focus on inner interpretability tools. We introduce a taxonomy that classifies methods by what part of the network they help to explain (weights, neurons, subnetworks, or latent representations) and whether they are implemented during (intrinsic) or after (post hoc) training. To our knowledge, we are also the first to survey a number of connections between interpretability research and work in adversarial robustness, continual learning, modularity, network compression, and studying the human visual system. We discuss key challenges and argue that the status quo in interpretability research is largely unproductive. Finally, we highlight the importance of future work that emphasizes diagnostics, debugging, adversaries, and benchmarking in order to make interpretability tools more useful to engineers in practical applications.

Machine Learning,Artificial Intelligence,Computation and Language,Computer Vision and Pattern Recognition
Interpretability of deep learning models: A survey of results

Supriyo Chakraborty,Richard Tomsett,Ramya Raghavendra,Daniel Harborne,Moustafa Alzantot,Federico Cerutti,Mani Srivastava,Alun Preece,Simon Julier,Raghuveer M. Rao,Troy D. Kelley,Dave Braines,Murat Sensoy,Christopher J. Willis,Prudhvi Gurram

DOI: https://doi.org/10.1109/uic-atc.2017.8397411

2017-08-01

Abstract:Deep neural networks have achieved near-human accuracy levels in various types of classification and prediction tasks including images, text, speech, and video data. However, the networks continue to be treated mostly as black-box function approximators, mapping a given input to a classification output. The next step in this human-machine evolutionary process — incorporating these networks into mission critical processes such as medical diagnosis, planning and control — requires a level of trust association with the machine output. Typically, statistical metrics are used to quantify the uncertainty of an output. However, the notion of trust also depends on the visibility that a human has into the working of the machine. In other words, the neural network should provide human-understandable justifications for its output leading to insights about the inner workings. We call such models as interpretable deep networks. Interpretability is not a monolithic notion. In fact, the subjectivity of an interpretation, due to different levels of human understanding, implies that there must be a multitude of dimensions that together constitute interpretability. In addition, the interpretation itself can be provided either in terms of the low-level network parameters, or in terms of input features used by the model. In this paper, we outline some of the dimensions that are useful for model interpretability, and categorize prior work along those dimensions. In the process, we perform a gap analysis of what needs to be done to improve model interpretability.
Adversarial Attack and Interpretability of the Deep Neural Net-Work from the Geometric Perspective

Mengfei XIA,Zipeng YE,Wang ZHAO,Ran YI,Yongjin LIU

DOI: https://doi.org/10.1360/ssi-2020-0169

2021-01-01

Abstract:Deep learning has achieved significant success in various engineering fields. However, its drawback has also received considerable attention recently, i.e., it suffers from poor interpretability, weak robustness and difficulty for network training, which seriously affect the security and usability of deep neural networks. Therefore adversarial attacks and interpretability become the focuses of the next generation of artificial intelligence research. In this paper, we survey recent works on them from a novel geometric perspective. We reformulate the problems in traditional deep learning models from the viewpoint of manifold theory, and summarize several strategies for possible optimization of the deep networks based on interpretability. Finally, we state several challenges on the interpretability from manifold theory and outline possible future directions.
DISCOVER: Making Vision Networks Interpretable via Competition and Dissection

Konstantinos P. Panousis,Sotirios Chatzis

DOI: https://doi.org/10.48550/arXiv.2310.04929

2023-10-08

Abstract:Modern deep networks are highly complex and their inferential outcome very hard to interpret. This is a serious obstacle to their transparent deployment in safety-critical or bias-aware applications. This work contributes to post-hoc interpretability, and specifically Network Dissection. Our goal is to present a framework that makes it easier to discover the individual functionality of each neuron in a network trained on a vision task; discovery is performed in terms of textual description generation. To achieve this objective, we leverage: (i) recent advances in multimodal vision-text models and (ii) network layers founded upon the novel concept of stochastic local competition between linear units. In this setting, only a small subset of layer neurons are activated for a given input, leading to extremely high activation sparsity (as low as only $\approx 4\%$). Crucially, our proposed method infers (sparse) neuron activation patterns that enables the neurons to activate/specialize to inputs with specific characteristics, diversifying their individual functionality. This capacity of our method supercharges the potential of dissection processes: human understandable descriptions are generated only for the very few active neurons, thus facilitating the direct investigation of the network's decision process. As we experimentally show, our approach: (i) yields Vision Networks that retain or improve classification performance, and (ii) realizes a principled framework for text-based description and examination of the generated neuronal representations.

Computer Vision and Pattern Recognition,Machine Learning
Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016)

Been Kim,Dmitry M. Malioutov,Kush R. Varshney

DOI: https://doi.org/10.48550/arXiv.1607.02531

2016-07-28

Abstract:This is the Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), which was held in New York, NY, June 23, 2016. Invited speakers were Susan Athey, Rich Caruana, Jacob Feldman, Percy Liang, and Hanna Wallach.

Machine Learning
Neural network interpretability with layer-wise relevance propagation: novel techniques for neuron selection and visualization

Deepshikha Bhati,Fnu Neha,Md Amiruzzaman,Angela Guercio,Deepak Kumar Shukla,Ben Ward

2024-12-07

Abstract:Interpreting complex neural networks is crucial for understanding their decision-making processes, particularly in applications where transparency and accountability are essential. This proposed method addresses this need by focusing on layer-wise Relevance Propagation (LRP), a technique used in explainable artificial intelligence (XAI) to attribute neural network outputs to input features through backpropagated relevance scores. Existing LRP methods often struggle with precision in evaluating individual neuron contributions. To overcome this limitation, we present a novel approach that improves the parsing of selected neurons during LRP backward propagation, using the Visual Geometry Group 16 (VGG16) architecture as a case study. Our method creates neural network graphs to highlight critical paths and visualizes these paths with heatmaps, optimizing neuron selection through accuracy metrics like Mean Squared Error (MSE) and Symmetric Mean Absolute Percentage Error (SMAPE). Additionally, we utilize a deconvolutional visualization technique to reconstruct feature maps, offering a comprehensive view of the network's inner workings. Extensive experiments demonstrate that our approach enhances interpretability and supports the development of more transparent artificial intelligence (AI) systems for computer vision applications. This advancement has the potential to improve the trustworthiness of AI models in real-world machine vision applications, thereby increasing their reliability and effectiveness.

Neural and Evolutionary Computing,Artificial Intelligence,Computer Vision and Pattern Recognition,Machine Learning
Unwrapping The Black Box of Deep ReLU Networks: Interpretability, Diagnostics, and Simplification

Agus Sudjianto,William Knauth,Rahul Singh,Zebin Yang,Aijun Zhang

DOI: https://doi.org/10.48550/arXiv.2011.04041

2020-11-09

Abstract:The deep neural networks (DNNs) have achieved great success in learning complex patterns with strong predictive power, but they are often thought of as "black box" models without a sufficient level of transparency and interpretability. It is important to demystify the DNNs with rigorous mathematics and practical tools, especially when they are used for mission-critical applications. This paper aims to unwrap the black box of deep ReLU networks through local linear representation, which utilizes the activation pattern and disentangles the complex network into an equivalent set of local linear models (LLMs). We develop a convenient LLM-based toolkit for interpretability, diagnostics, and simplification of a pre-trained deep ReLU network. We propose the local linear profile plot and other visualization methods for interpretation and diagnostics, and an effective merging strategy for network simplification. The proposed methods are demonstrated by simulation examples, benchmark datasets, and a real case study in home lending credit risk assessment.

Machine Learning,Artificial Intelligence

Proceedings of AAAI 2019 Workshop on Network Interpretability for Deep Learning

Visual Interpretability forDeepLearning

A Survey on Neural Network Interpretability

Visual Interpretability for Deep Learning: a Survey

Functional Network: A Novel Framework for Interpretability of Deep Neural Networks

A Comprehensive Review of Deep Neural Network Interpretation Using Topological Data Analysis

Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond

Demystifying Deep Learning in Networking

On Interpretability of Artificial Neural Networks: A Survey

A Survey of the Interpretability Aspect of Deep Learning Models

Interpreting AI for Networking: Where We Are and Where We Are Going

Feature Analysis Network: An Interpretable Idea in Deep Learning

Interpretable neural networks: principles and applications

Proceedings of ICML 2021 Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI

Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks

Interpretability of deep learning models: A survey of results

Adversarial Attack and Interpretability of the Deep Neural Net-Work from the Geometric Perspective

DISCOVER: Making Vision Networks Interpretable via Competition and Dissection

Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016)

Neural network interpretability with layer-wise relevance propagation: novel techniques for neuron selection and visualization

Unwrapping The Black Box of Deep ReLU Networks: Interpretability, Diagnostics, and Simplification