Targeted Background Removal Creates Interpretable Feature Visualizations

Ian E. Nielsen,Erik Grundeland,Joseph Snedeker,Ghulam Rasool,Ravi P. Ramachandran
2023-06-23
Abstract:Feature visualization is used to visualize learned features for black box machine learning models. Our approach explores an altered training process to improve interpretability of the visualizations. We argue that by using background removal techniques as a form of robust training, a network is forced to learn more human recognizable features, namely, by focusing on the main object of interest without any distractions from the background. Four different training methods were used to verify this hypothesis. The first used unmodified pictures. The second used a black background. The third utilized Gaussian noise as the background. The fourth approach employed a mix of background removed images and unmodified images. The feature visualization results show that the background removed images reveal a significant improvement over the baseline model. These new results displayed easily recognizable features from their respective classes, unlike the model trained on unmodified data.
Computer Vision and Pattern Recognition,Artificial Intelligence,Human-Computer Interaction
What problem does this paper attempt to address?
The paper aims to address the issue of the difficulty in understanding the feature visualization results of deep neural networks. Specifically, the authors found that unregularized feature visualization outputs often contain a large amount of high-frequency noise, making it difficult for humans to identify the key features. To solve this problem, the paper proposes a method to improve feature visualization by changing the background of the training data. The specific methods include: 1. **Basic Model**: Train with unmodified data. 2. **Background Removal Model**: Remove the image background during training so that the model focuses only on the main object. 3. **Gaussian Noise Background Model**: Replace the background with Gaussian noise. 4. **Mixed Dataset Model**: Part of the training data has the background removed, while the other part remains unchanged. Experimental results show that the training method of removing the background can significantly improve the interpretability of feature visualization, making the generated feature maps easier for humans to recognize. This method not only enhances the transparency of the model but also may help increase user trust in the model and reduce the risk of misidentification in practical applications.