Can We Trust Deep Learning Models in SAR ATR?

Yuze Gao,Weiwei Guo,Dongying Li,Wenxian Yu
DOI: https://doi.org/10.1109/lgrs.2024.3356727
IF: 5.343
2024-02-10
IEEE Geoscience and Remote Sensing Letters
Abstract:Deep learning has significantly enhanced the performance of automatic target recognition (ATR) in synthetic aperture radar (SAR). However, the concept of model overinterpretation, characterized by classifiers discerning strong class evidence within image regions that lack semantically salient features related to targets (e.g., background clutter in SAR images), has undermined confidence in the reliability of deep learning models. Previous studies predominantly relied solely on one interpretability method to qualitatively identify the key input pixels, without assessing the efficiency of these features in decision-making process, posing a significant hurdle in evaluating the model overinterpretation. In this letter, we propose necessity–sufficiency index (NSI) to select models' decision-making basis among all the key regions identified by multiple interpretability methods and segmentation algorithm. Furthermore, we propose weighted composition ratio statistic (WCRS) method to quantitatively analyze the model overinterpretation by incorporating the NSI as weighted average weights. The experimental results indicate that our methods are capable of accurately identifying the decision-making features and quantitatively analyzing the models' tendency toward overinterpretation.
imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics
What problem does this paper attempt to address?