Abstract:Physical experiments often involve multiple imaging representations, such as X-ray scans and microscopic images. Deep learning models have been widely used for supervised analysis in these experiments. Combining different image representations is frequently required to analyze and make a decision properly. Consequently, multi-view data has emerged - datasets where each sample is described by views from different angles, sources, or modalities. These problems are addressed with the concept of multi-view learning. Understanding the decision-making process of deep learning models is essential for reliable and credible analysis. Hence, many explainability methods have been devised recently. Nonetheless, there is a lack of proper explainability in multi-view models, which are challenging to explain due to their architectures. In this paper, we suggest different multi-view architectures for the vision domain, each suited to another problem, and we also present a methodology for explaining these models. To demonstrate the effectiveness of our methodology, we focus on the domain of High Energy Density Physics (HEDP) experiments, where multiple imaging representations are used to assess the quality of foam samples. We apply our methodology to classify the foam samples quality using the suggested multi-view architectures. Through experimental results, we showcase the improvement of accurate architecture choice on both accuracy - 78% to 84% and AUC - 83% to 93% and present a trade-off between performance and explainability. Specifically, we demonstrate that our approach enables the explanation of individual one-view models, providing insights into the decision-making process of each view. This understanding enhances the interpretability of the overall multi-view model. The sources of this work are available at: <a class="link-external link-https" href="https://github.com/Scientific-Computing-Lab-NRCN/Multi-View-Explainability" rel="external noopener nofollow">this https URL</a>.

Studying the Impact of Latent Representations in Implicit Neural Networks for Scientific Continuous Field Reconstruction

Continuous Field Reconstruction from Sparse Observations with Implicit Neural Networks

DeepISMNet: three-dimensional implicit structural modeling with convolutional neural network

IDLat: An Importance-Driven Latent Generation Method for Scientific Data

Deep Learning Improvements for Sparse Spatial Field Reconstruction

Latent Communication in Artificial Neural Networks

Continuous Spatial-Spectral Reconstruction Via Implicit Neural Representation

Enriching Point Clouds with Implicit Representations for 3D Classification and Segmentation

LatentExplainer: Explaining Latent Representations in Deep Generative Models with Multi-modal Foundation Models

Effective Latent Differential Equation Models via Attention and Multiple Shooting

Learning continuous shape priors from sparse data with neural implicit functions

Towards Comparative Physical Interpretation of Spatial Variability Aware Neural Networks: A Summary of Results

Explainable Multi-View Deep Networks Methodology for Experimental Physics

Multi-scale Iterative Model-guided Unfolding Network for NLOS Reconstruction.

Exploring Deep Registration Latent Spaces

ReFiNe: Recursive Field Networks for Cross-modal Multi-scene Representation

Implicit neural representation for change detection

Where Do We Stand with Implicit Neural Representations? A Technical and Performance Survey

Continual Neural Mapping: Learning an Implicit Scene Representation from Sequential Observations

Explaining the Implicit Neural Canvas: Connecting Pixels to Neurons by Tracing their Contributions

Efficient Implicit Neural Reconstruction Using LiDAR