Abstract:The rapid development of deep learning has greatly improved the performance of various computer vision tasks. However, the "black box" nature of deep learning network models makes it difficult for users to understand its decision-making mechanism, which is not conductive to model structure optimization and security enhancement and also greatly increases the training cost. Focusing on the task of intelligent image interpretation, this paper makes a comprehensive review and comparison of the research progress of deep learning interpretability. Firstly, we group the current interpretability analysis methods into six categories: activation maximization method, surrogate model, attribution method, perturbation-based method, class activation map based method and example-based method, and review the principle, focus, advantages, and disadvantages of existing related works. Secondly, we introduce eight evaluation metrics that measure the reliability of the explanations provided by the various interpretability analysis methods, and sort out the current publicly available open source libraries for deep learning interpretability analysis. Based on the open source library, we verify the applicability of the current deep learning interpretability analysis methods to the interpretation of remote sensing images. The experimental results show that the current interpretability methods are applicable to the analysis of remote sensing interpretation, but have certain limitations. Finally, we summarize the open challenges of using existing interpretability algorithms for remote sensing data analysis, and look forward to the prospect of designing interpretability analysis methods oriented to remote sensing images. We hope this review can promote the research on interpretability methods for remote sensing image interpretation, so as to provide reliable theoretical support and algorithm design guidance for the application of deep learning technology in remote sensing image interpretation tasks.

Discovering interpretable models of scientific image data with deep learning

Looking deeper into interpretable deep learning in neuroimaging: a comprehensive survey

Visual Interpretability forDeepLearning

Interpretability of deep learning models: A survey of results

A Survey of the Interpretability Aspect of Deep Learning Models

Visual Interpretability for Deep Learning: a Survey

Explainable Deep Learning in Spectral and Medical Image Analysis

Deep Learning Interpretability Analysis Methods in Image Interpretation

Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models

TorchEsegeta: Framework for Interpretability and Explainability of Image-Based Deep Learning Models

Interpretable 2D Vision Models for 3D Medical Images

Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond

Deeper Interpretability of Deep Networks

Visual Interpretable and Explainable Deep Learning Models for Brain Tumor MRI and COVID-19 Chest X-ray Images

Multimodal Deep Learning for Scientific Imaging Interpretation

Model-Based Explainable Deep Learning for Light-Field Microscopy Imaging

Topological Interpretability for Deep-Learning

Visual interpretability of image-based classification models by generative latent space disentanglement applied to in vitro fertilization

Interpretable deep learning in single-cell omics

Towards Understanding Sensitive and Decisive Patterns in Explainable AI: A Case Study of Model Interpretation in Geometric Deep Learning

Interpretability of Machine Learning Methods Applied to Neuroimaging