Abstract:Introduction: The human brain processes shape and texture information separately through different neurons in the visual system. In intelligent computer-aided imaging diagnosis, pre-trained feature extractors are commonly used in various medical image recognition methods, common pre-training datasets such as ImageNet tend to improve the texture representation of the model but make it ignore many shape features. Weak shape feature representation is disadvantageous for some tasks that focus on shape features in medical image analysis. Methods: Inspired by the function of neurons in the human brain, in this paper, we proposed a shape-and-texture-biased two-stream network to enhance the shape feature representation in knowledge-guided medical image analysis. First, the two-stream network shape-biased stream and a texture-biased stream are constructed through classification and segmentation multi-task joint learning. Second, we propose pyramid-grouped convolution to enhance the texture feature representation and introduce deformable convolution to enhance the shape feature extraction. Third, we used a channel-attention-based feature selection module in shape and texture feature fusion to focus on the key features and eliminate information redundancy caused by feature fusion. Finally, aiming at the problem of model optimization difficulty caused by the imbalance in the number of benign and malignant samples in medical images, an asymmetric loss function was introduced to improve the robustness of the model. Results and conclusion: We applied our method to the melanoma recognition task on ISIC-2019 and XJTU-MM datasets, which focus on both the texture and shape of the lesions. The experimental results on dermoscopic image recognition and pathological image recognition datasets show the proposed method outperforms the compared algorithms and prove the effectiveness of our method.

Shape-Texture Debiased Neural Network Training

Shape-Biased Learning by Thinking Inside the Box

ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness

The Origins and Prevalence of Texture Bias in Convolutional Neural Networks

Emergence of Shape Bias in Convolutional Neural Networks through Activation Sparsity

Shape-biased CNNs Are Not Always Superior in Out-of-Distribution Robustness

Robustifying Deep Vision Models Through Shape Sensitization

Informative Dropout for Robust Representation Learning: A Shape-bias Perspective

Teaching CNNs to Mimic Human Visual Cognitive Process & Regularise Texture-Shape Bias

On the Influence of Shape, Texture and Color for Learning Semantic Segmentation

Teaching deep networks to see shape: Lessons from a simplified visual world

3D Shape Contrastive Representation Learning with Adversarial Examples

Reducing Texture Bias of Deep Neural Networks via Edge Enhancing Diffusion

Edge Enhancement Improves Adversarial Robustness in Image Classification

Clarifying Myths About the Relationship Between Shape Bias, Accuracy, and Robustness

Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study

STNet: shape and texture joint learning through two-stream network for knowledge-guided image recognition

BiasBed -- Rigorous Texture Bias Evaluation

Temporally Resolution Decrement: Utilizing the Shape Consistency for Higher Computational Efficiency

Adversarial Training of Deep Neural Networks Guided by Texture and Structural Information

InBiaseD: Inductive Bias Distillation to Improve Generalization and Robustness through Shape-awareness